CCTC-Linux

Show local scope variables.

Show global variables.

https://cs.lmu.edu/~ray/notes/regex/

Shows different grep regex uses.

man grep | grep -EA3 "(\-E,|\-P,)"

Extended grep -Both commands accept the full set of regular expression capabilities.

Example- find .-regextype sed -regex "{some regex}"

Example- locate -regex "{some regex}"

Any single character one time.

Any characters/wildcard.

May or may not appear {applies to preceding character set or character}.

One or more of preceding character set or character.

Performs some system integrity checksSearches, loads, and executes boot loader programLoads and executes MBR

Located in 1st sector of bootable disk: /dev/hda or /dev/sdaLess than 512 bytes in sizePrimary boot loader 1st 446 bytesPartition table nes 64 bytesMBR validation check last 2 bytesContains GRUB (LILO for old sytems)Loads and executes GRUB boot loader.

GRUB-Grand Unified Bootloader (Loads Kernel into RAM)Can choose kernel images during bootup.Displays a splash screen, waits for a few seconds.Has the knowledge of filesystems.Configuration file is /boot/grub/grub.conf

Menu drive boot menu.Can boot different OS's depending on default setting or choose an OS to loadControlled via a config file (location depends on distro, /boot/grub.conf or /boot/grub/grub.cfg or /boot/grub/menu.lst

Much different than GRUB 1.xConfig file is grub.cfgInstall command-gub-installConfiguration command: grub-mkconfigConfiguration options: /etc/default/grub

Stage 0 (process 0)Establishes memory Management, detects CPU type, page tables.Mount initrd and unpacks initramfs from it.Initial RAM Disk (initrd) is used by kernel as temp root file system until kernel is booted and the root file system is mounted. Contains necessary drivers complied inside, which helps to access the hard drive partitions, and other hardware.Mounts root file system as specified in the "root=" in grub.conf (grub 2.X) menu list (grub 1.X).Kernel executes the /sbin/init program by default.Mounts initramfs as initial root file systeminitramfs loads drivers, mounts actual root file system as passed in root= by bootloaderinitramfs runs /sbins/init

-/sbin/initParent process of all user space programs, always PID of 13 main initialization processes.********************Initialization Process 1*****************System V init:Based on run levels2 Primary components/etc/inittab file-determines the initial runlevel for the system to boot into.-runlevel determined by initdefault option-application run for applicable runlevelsless/etc/inittabbootscripts-/etc/init.d/rc-Script that runs Start and Kill scripts out of applicable rc foldero file /etc/init.d/rco less /etc/init.d/rc*********************Initialization Process 2*******************System D-Initiated in 2010 to create a service manager for Linux-Includes device management (udevd) and logging (journald)-/sbin/init symbolically linked to /lib/systemd/systemdo runs the /etc/systemd/default.target which is a symbolically link to desired initial traget in /lib/systemd/systemo target creates a dependency tree which calls other target scriptso multiple systemd programs available to handle-uses the terms targets and unitso target is essentially equivalent to runlevelo unit is essentially equivalent to daemon (service)*******************Initialization Process 3*********************-upstarto Created for Ubuntu desktops (Ubuntu 15.04+ are systemd)o Event driven, starts jobs based on eventso Configuration files for jobs kept in /etc/init/o Continuous monitoring of the system after startup

Hard disks utilizing a MBR can have a maximum of 4 primary partitions and a max size of 2 TB each.Typically the MBR is located on /dev/hda or /dev/sdaIt contains information about GRUB.Extended partitions overcame the 4 primary issue by moving addressing for additional partiions into the extended partition itself rather than in the MBR table.The fdisk utility is used to manage MBR disk partitions.

Up to 128 partitionsPartition sizes over 2TBThe gdisk utility is used to manage GPT partitions-Logical Volume Management (LVM)

Linux utilizes LVM physical volumes to create pools of storage known as volume groups.volume groups can be expanded without repartitoning the underlying disk.Installation of Linux requires a minimum of one partition containing the / (root) directory. Typical installation will include a partition for swap space as well which is an extension of Linux memory, however, this swap space can exist in a swap file as well.The /etc/fstab configuration file is used to manage the mounting of partitions at boot time.

Start init is the parent/grandparent of all processes. Its process ID is 1, and its parent is 0. When the kernel starts init, it searches for program /sbin/init. Can be overridden using kernel command line option init=.SystemD based distros have a different init system. The kernel still looks for /sbin/init, but SystemD systems typically a symbolic link to /lib/systemd/system/{name}.service. Like init, its process ID is 1, and its parent is 0.Modern Linux kernels/distros have kthreadd which is a kernel thread daemon which is second after init, it has a PID of 2 and kernel/PID 0 as its parent.kthreadd manages hardware and is directly handled by the kernel (kernel mode) and has a high priority. Are identified bye square brackets [kthreadd].kkthreadd spawned processes will have a PPID of 2.Mount filesystems in /etc/fstab.Set system time from time stored in hardware.Supported filesystems in: /proc/filesystems

e=every processf=file

l=long list--forest=ASCII forest

Resource: CPU and Memory usage process stats

o=format

forest view (BSD style)

Orphaned process waiting for an event in the background.stdin, stdout, and stderr are pointed to /dev/nullAll Daemons are Orphans, but all Orphans are not necessarily DaemonsA Daemons purpose is to manage/monitor a service: {status, start, restart, stop}Works on most systems:cd /etc/init.d/./servicename.start./servicename.restart./servicename.stopNon-systemd / SysVinit systemsservice {servicename}{status.start.restart.stop}systemdsystemctl {status.start.restart.stop}{servicename}

Reads a series of commands and executes them at a later time

parent process exited, adopted by /sbin/init (PPID is 1)Display total number of PID'scat /proc/sys/kernel/pid_max

Process has exited, waiting on parent to acknowledge or terminate.Can't be terminated with kill since it has already finished execution, unless the parent process that spawned it is killed.Zombies can be a threat as too many could utilize all available PID's, and/or system resources, and crash.

Additional Characters< :high-priority (not nice to other users)N :low-priority (nice to other users)L :has pages locked into memory (for real-time and custom IO)s :is a session leaderl :is multi-threaded+ :is in the foreground process group

find /var/log -maxdepth 1 -type fLogs are managed by syslog daemon systemctl status rsyslog.service grep rsyslog /etc/rsyslog.conf

Applications can choose to handle their own logging, or to use the syslog or journald logging applications to handle their logs.In most cases applicatoins will default to /var/log for log storage, unless they have a it specifically specified in the config file to use another resource.

A&A logs are almost always system logs; there are very few cases where this is not the case.A&A logs differ from system logs, as system logs only deal with the kernel, and auth logs deal with both the kernel and the service attempting to authenticate.

Logrotate is aprogram that can manage your local log filesUse cron to run logrotateThe main configuration file contains configuration settings for all logs is /etc/logrotate.confIndividual configuration files are kept in the /etc/logrotate.d directory

RFC5424-Syslog protocol/etc{syslog, rsyslog}.conf settings file to determine how to handle messages.Default location for log files on most Linux variants is /var/logCapable of sending log messages to a networked syslog server (udp/tcp514)Reads messages off the /dev/log domain socket

rsyslog rules consists of a filter and an action on a single lineFilter is written to match items of interest in a messageAction is performed if the filter matchesFacility/Priority-based filters (selectors) filter syslog messages using facility and priority separated by a period and have the form: FACILITY.PRIORITYApplication syslog componentsFacility is a code used to identify the program type that generates the message for the log. RFC 5424 defines numerical codes for the facilitiesRulescron.*,*.err-asterisk to specify all facilities or prioritiescron,lpr,=notice,=warning-commas to separate names of multiple facilities and priorities*.notice;lpr.none;authpriv.none:cron.none-semicolons to separate multiple filters on the same linelpr.!info.!debug-exclamation point before a priority to match all messages except those with that priorityauthpriv.none-none for a priority to not redirect messages for that facilityRule Exampleskern.*-match all kern messages with any prioritymail.crit-match all mail messages with crit or higher prioritydaemon.!info.!debug-match all daemonmessages except those with info or debug prioritymail.=info-match mail messages only with .info prioritylpr.mail.=warning,=err-match all lpr and mail messages with warning or err priority

ls -la /dev/log-This is a domain socket typels -la /proc/kmsg-Kernel ring bufferdmesg-Read from kernel ring buffer, non-persisted (Look here for buffer overflow info)

Journald doesn't use Syslog faciliteis by default (can be enabled in conf). Journald is designed to work with PIDs, Process Names, and Service IDs. It does use severtiy codes.Part of systemdConfig file located at /etc/system/journald.confWorks with rsyslog to record log files - controled by the ForwardToSyslog setting in journald.confCan be configured to persist log files itself, default in /var/log/journal - controlled by Storage setting in journald.confOffers fine grain querying of journal log via journalctl command

auditd is the userspace component to Linux Auditing System.Linux Auditing System operates at the kernel levelAudit rules by default kept in /etc/audit/auditd.rules, config in /etc/audit/auditd.confDetermines how the audit system functions once the daemon has been started.auditd has been integrated into systemd on modern distros

Debian /var/log/auth.log or Red Hat /var/log/secure-store info about logins./var/log/btmp-log tracks failed login attempts and is not human readable.Commands to uselast -f /var/log/btmp-follow bad login attempts real timelastb - -shows all the bad logins attempts in btmp/var/run/utmp-log tracks who is currently using the system and is not human readableCommands to uselast -f /var/log/utmp-follows logged in userswho -shows who is logged into the system and how/var/log/wtmp-log tracks history of which users have logged into and out of the system and is not human readable.-can be used to determine when system was last rebooted or shutdown and by who.-identifies the tty user logged into and whether it was a local login or not and the remote IP used to connectCommands to uselast-show most recently logged in users and if they are still connectedlast -x -show system shutdown entries and run level changes./var/log/lastlog-database file recording the last login for each account.Commands to uselastlog -records last time account logged into system (console logins only)

Multiple system calls available to programs to create syslog or journald log entries.syslog() via libcsd_journal_print() via Journal APIStandard output and error of system servicesKernel ring bufferAudit records via audit subsystem

List of hosts and associated IP's.Central file that controls resolver setup (host.conf). Resides in /etc, tells resolver which services to use and in what order.

Name Server settings.File that manages nameserver information.

Determines order of precedence.Determines default order of precedence for DNS resolution.

Network socket is an internal endpoint for sending or receiving data at a single node in a computer network.Typical network socket format: (IP:Port): 1.2.3.4:25

Protocol stack processes its respective layer, performing address, checksum validation, removes its respective header and trailer and passes up the content to its immediate upper layer.

No layer checking done, it is up to the application using the raw socket ot interpret the data. RAW sockets are used as packet capture/sniffer programs as it captures "raw" data from the network interface card and passes it directly to the application.

Print network connection, routing tables, interface statistics, masquerade connections, and multicast memberships.

DNS lookup utility.

Interrogates DNS name servers.

Query internet name servers interactively.

Listen for network connection on behalf of another program.Hands off control of that connection to intended server.Help reduce memory load and improve security.There are two different super servers:inetd uses TCP wrappers to handle securityxinetd has built in security featuresServers that normally use super server: telnet, FTP, TFTP, rlogin, finger, POP, IMAP.

bind= <IP Address>: listens only on network interface for the service.only_from=<IP Addresses|Network>: accept connections only from IP addresses.no_access=<IP Addresses|network>: Deny connections only from IP addresses.access_times=hour:min-hour:min: sets time which users may access the server.banner=/usr/local/etc/deny_banner

Allows windows file and printer sharing on LinuxfreeUses CIFS (Common Internet File System)Can create more overhead to configureCan be chanllenging to get working rightConfig file: /etc/samba/smb.conf

Can't transfer a file natively.Use SSH insteadNetcat can transfer a file.File transfer via telnet uuencode/base64 uudecode/base64 -d

Determining Shell Type (Shell options)shopt login_shell

Displays current shell.

Shell invoked from another shellStarted by a program without a login

Shells running scripts, can run in background.Can't read Standard Input (STDIN)

Interactive: reads commands from user input on a tty ($PS1 prompt variable is set)

Input into a program, file descriptor 0Represented as 0< or < when redirecting standard input from a file

Reads commands from user input.

Runs in the background.

Input into a program, file descriptor 0Represented as 0< or < when redirecting standard input from a file

Output of a program, file descriptor 1Represented as 1> or > when redirecting standard output to a file

Output of a programs error handler, file descriptor 2Represented as 2> when redirecting standard error to a file

-to append rather than overwrite when redirecting.-when redirecting standard error into standard output.

Uni-directionalExists in RAMOpened at time of creation

Created with mkfifo command or mknod p commandExit on filesystem with a nameCan be accessed by unrelated processes.Bi-directional(FIFO, First In First Out)Creates Named Pipes.A named pipe has a file name on your file system and can be accessed by independent processes that were not spawned by the same parent process.

Shell reads commands from a file (script) or directly from the user's terminal.Initial job of the parsing engine is lexical analysis: to separate the stream of characters into words and apply meaning to the results. The word (token) is the base unit which the parser operates. Words are sequences of characters separated by metacharacters, which include simple separators like spaces and tabs, or characters that are special to the shell language, like semicolons and ampersands. Alias Expansion is performed at this step.Shell parses (analyzes and substitutes) the tokens into simple (cat file) and compound commands (if..then..fi).Bash performs various shell expansions, breaking the expanded tokens into list of filenames and commands and arguments.Brace expanision-echo {1...3}Tilde expansion is used to expand to several specific pathnames.home directories: echo ~current working directory: echo ~+previous working directory: echo~-Parameter and variable expansionword="car" echo $word echo ${word}scommand substitution'command' $(command)thedate=$(date)Arithmetic expansiona='expr $z + 3'a=$(($z+3))Work splittingIFS-Internal Field Separator <SPACE><TAB<>NEWLINE>After (Parameter, command, and arithmetic) expanision bash scans the results of IFSs.The IFS variable holds characters that bash sees as a word boundaries.Filename expansionecho "hello" > Middletoncat Midd*Redirection is performed if necessary, redirection operators and there operands are removed from the argument list.Commands are xecuted.Shell functionsBuiltin commandsHash tablePath VariableOptionally the shell waits for the command to complete and collects its exit status.

>ls /bin | grep netcat>file /bin/netca*

which {nc,netcat}, evaluates the PATH in order and displays the first instance of the command if it existswhich -a{command}, evaluates the PATH in order and displays all instances of the command if it exists.

hash, displays bash's hash table (Recently executed commands)hash -r, clear bash's hash table

type {Command}, evaluates bash order of evaluation to determine the first instance of the commandtype -a{command}, shows all instances of command in the order of bash evaluation

echo $HISTFILE/home/username/.bash_historyecho $HISTFILESIZE2000 (example)echo $HISTSIZE1000 (example)

max lines in history

Determines the amount of lines saved to ~/.bash_history upon session exit.

list the processes running on the system

send a signal to noe or more processess to stop itKill 1 or %1 (saved job)

an alternate way of listing your own processes

put a process in the backgroundTo put in the background hit CTRL-Z%1

put a process in the foreground

Semi-Colon ;Executes next command unconditionally / regardless of the xit statusExample: ping -c4 127.0.0.1; echo "All done"Logical/Operator &&Executes next command only if the previous command succeeded / exit status of 0Example: cd /tmp is && echo "it works"cd /fakedire && ls && echo "It works"

Displays the manual for a command, automatically paged using more:Example: man -s 7 {command}man -k {command}manpath | tr':' '\in'apropos-Each manual page has a short description, searches for keywords. (Non-Built in commands)

Displays the most used switches/options for the chosen command

if in $PATHView binary path AND manpage path

Search possibly compressed files for a regular expression.

Search possibly compressed files for a regular expression.

Returns EVERY file name containing "find" in the filename; case insensitive.

only if it exists

Information about processes, connections and some hardware.

Information about the system (hardware)

Information about processes, connections, and some hardware.

Device directory that is dynamically populated by udev (FKA devfs)

df -t, filesystem disk usage.find / -maxdepth 1 -type d, filesystem information (main)

File System Information (main)find / -maxdepth -type d

Appears as a mounted volume, but is actually stored in volatile memory/var/run on older distributions, /run on newer.

Pseudo file system providing information about kernel, hardware, device drivers.Designed to add structure to the old procfs systems

mtime: Modified time - time file content was last modifiedatime: Access time - time the file was last accessed (persistent for 24hrs)ctime: Change time - time of inode record change (file attribute changes: size, location, type, etc.)

A Unix file is "stored" in two different parts of the disk - the data blocks and the inodesThe data blocks contain the "contents" of the file. The information about the file is stored elsewhere - in the inode.The inode contains the following pieces of information:Owner IDGroup IDSize of fileNumber of hard links to the fileTime last accessedTime last modifiedTime inode last modifiedls -i: list the inode of a fileAn iode is a data structure on a filesystem on Linux and other Unix-like operating systems that stores all the information about a file except its name and its actual data.http://www.linfo.org/inode.html

crtime: creation time

ls -la-Show owner:groupchown-Change ownerchown {owner}:{group}{file}-Usagechgrp {group}{file}-Usagegetent gets global config filesgetent -help, get entries from Name Service Switch librariesgetent group, show all the groupsgetent passwordgetent {group, sudo), show sudo group output only

Program with SUID runs under the security context of the user (owner) of the programUppercase "S" indicates the execute bit IS NOT set, lowercase "s" execute bit IS set.

Program set with SGID runs under the security context of the group of the program. Uppercase "S" indicates the execute bit IS NOT set, lowercase "s" execute bit IS set.

When set on a directory only the owner of the file can delete or rename files in that directory. (used for directory only)Uppercase "T" indicates the execute bit IS NOT set, lowercase "t" execute bit is set.

When bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readableWhen an interactive shell that is not a login shell is started, bash reads and executes commands from /etc/bash.bashrc and ~/.bashrcPersistenceAlso applies to any aliases, functions, scripts, etc. running from these file locations.

file applies only to interactive BASH shell

File applies only to Bourne and BASH compatible shells; SETS $PATH VAR:echo $PATH | tr':''\n'

files applies only to BASH non-login shells

File applies only to login shells.

clarifies the interactive vs. non-interactive shell env. files

Some crons use an extended cron which as Y after day of the week in the range of 1900-3000.Edit your own cron file /var/spool/cron/crontabscrontab -e opens user crontabe in a text editorSystem cron entries ls -d /edtc/cron*https://crontab.guru/Minute, Hour, Day of the Month, Month of the Year, Month of the Week.

/etc/inittabDefault runlevel is determined by the initdefault labelid:5:initdefaultEach line in inittab has an actionsysint-executed at system bootrespawn-restared when terminated (gettys)once-executed a single time when run level enteredwait-executed when run level is entered, int waits for termination to continueLines in the file are borken into:id:rstate:action:processs2:23:wait:/sbin/rc 2-id is s2, runlevels 2 or 3, wait action, execute '/sbin/rc/ 2'/etc/rc#.d foldersK and S scripts (uppercase, scripts beginning with a lowercase k or s are ignored)Scripts typically link back to scripts in /etc/init.d/When a runlevel is changed, first all 'K' scripts are executed in numeric order, then all 'S' scripts are runRunlevels 0(halt) and 6 (reboot) will only stop scripts, so 'S' scripts are run with the stop commandNot all Linux flavors of Sys V init follow this pattern, some only run K scripts when a lower runlevel is changed to.

Extension of Sys V Init, adds parallel service startup, and dependency based boot.

Start of post kernel boot processLooks at the /etc/inittab file to decide the Linux run level.Available run levels 0 through 6Init identifies the default init level from /etc/inittab and uses that to load all appropriate program.Execute 'grep initdefault /etc/inittab' on a sysv init system to identify the default run levelSet system time from time stored in harwareMount file systems defined in /etc/fstabStarts the configuration of the environment for the system/users.

Sysv Init Runlevel Programs/etc/rcX.d where X is the runlevelChanging Runleveltelinit 3-change to runlevel 3

Shuts down the system.

Mode for administrative task.

Does not configure network interfaces and does not export network services.

Starts the system normally.

For special purposes.

Start the system normally with appropriate display manager (with GUI), same as runlevel 3 + display manager.

systemctl get-default-default targetsystemctl list-dependencies-show tree of dependenciessystemctl list-units --type service-state of servicessystemctl lit-units --type target-state of targetssystemctl list-unit-files-show services and their startup states

Instead of runlevels System D uses terms called targets (runlevels) and units (services): default.target is the default runlevel (equivalent to initdefault in the inittabe).Links to the targets are in /etc/systemd/system.Targets and units in /usr/lib/systemd/system/.System D managed with systemctlsystemctl status {servicename} (ex ssh.service)systemctl stop {servicename}systemctl start {servicename}systemctl disable {servicename}systemctl enable {servicename}

Logs managed and viewed with journalctljournalctl -b-view logs since last bootjournalctl --since "207-01-10" --until "2017-01-11 03:00"

Simpler boot process as compared to the init.Systemd provides concurrent and parallel process of system boot so it ensures better boot speed.Processes are tracked using control groups, not by PID'sImproved ways to handle boot and services dependencies.Capability of system snapshots and restoreMonitoring of started services; also capable of restarting any crashed servicesIncludes systemd-login module to control user logins.Ability to add and remove componentsLow memory foot prints and ability for job scheduling.Journald module for event logging and systlogd module for system log.Protected services (ex: unable to kill auditd)

Popular in embedded devices,https://busybox.net/FAQ.htmlhttps://busybox.net/about.html

Installs programs from repositoryPackage management:Search and install packages:apt-cache search {text}apt-get install {packagename}

recognize unusual names-takes time and experienceServices/programs run from unusual directories-pwdx- gets current working directory of processls -l /proc/{PID}/cwdps auxwwweps -ef-get PID with:ls -l /proc/PID/exe-find the path and binary from where it was invoked.

Find user/process making connection or listening on port:lsof -RPni :{XX}-replace XX with port numberlsof -Pnp {PID}-list verbose process associated with the process.

Displays stats for PACKET, TCP, UDP, DCCP, RAW, and Unix domain sockets

List all, don't resolve port numbers, show programs.

Listening ports, don't resolve port numbers, show program.

List all TCP, don't resolve port numbers, show program.

List listen TCP, don't resolve port numbers, show program.