1. Approach

This docu shows some tips and tricks for shell and cmd programming

Why yet another docu as all existings?

3. History and available shell script languages

bash, POSIX

4. cmd line

Generally calling a command in a console or terminal (emulation) creates a new process started with the named application with given arguments. Usual the terminal waits till end of this process.

xterm

~ Opens a new terminal emulation in an extra window, and waits till it ends.

Writing & after the command does not wait for ending the process:

xterm &

~ The calling terminal writes the PID of the started process and is ready to get the next command.

Generally: It is exact the same if the command is part of a shell script, which is called and executed.

As examples for a command line three commands are used here:

echo  outputs the given arguments to stdout
ls    lists the content of a directory
read  reads stdin and writes to an environment variable

The examples shows some aspects of this commands, without complete explanation of their possibilities. This commands are explained eloborately in some web pages.

4.1. stdin, stdout, stderr

This are the three designations of so named pipe channels: The console or terminal can write texts and can read the input from keyboard during the command is executed. Note: If the command is executed not starting in a terminal, instead in another application, it has also these channels. But these channels are processed by specific stuff in the calling environment. The pipe channels are a property of running process, the terminal is only that what’s adapted to it.

The terminal is usual programmed in a kind, that only a full line is offered to stdin. It means if something is typed, it has only an effect if the ENTER key was pressed (which offers a newline character to the pipe).

The stdout is outputted immediately to the terminal, the 'new line' sets the output cursor to the next line, and scrolls if necessary. The stderr is similar. Sometimes stdout and stderr are outputted in different colors.

To prevent muddle stdout and stderr should be outputted as a whole line, running in the same thread. But this is (should be proper) controlled by the application.

4.2. redirection of stdin, stdout, stderr

In a terminal it is simple to redirect all three pipe channels to files or from and to other commands executed before and after.

4.3. Arguments for command line

Simple texts without spaces are provided as arguments for the command. How arguments are processed, it depends on the command.

But there are some specials:

  • Spaces separates arguments

  • Arguments should be written in quotation marks "ARGUMENT", then the text inside the quotation marks is the argument as given.

  • But other or better than in Windows-Batch-Script: Am argument variable $1 etc. does not contain the "" of the calling level. In Windows batch, %1 does contain, use in Windows %~1 to have the same effect.

  • $ENV will be replaced with the value of the named environment variable .

  • $(command "arguments") executes the command to produce a text in the script similar as $ENV, usable for example as argument value or even in echo etc.; from the stdout of the command.

  • * delivers all files and directories with their names given in the current directory. This preparation is done by the terminal program or even by shell script execution.

  • path/to/leading*trailing/path does adequates but with the given path. The path can be absolute (starting with `/) or relative starting from the current directory ($PWD).

4.3.1. * as argument

The effect of * is able to demonstrate with:

echo *

The echo command does nothing with arguments. It outputs the argument value only to stdout. Even that’s why you can see, the * expansion come from the command line interpreation, not from the command itself.

4.4. set environment variables to the called process

You can set some environment variables which are valid only for the called process in the same command line:

ENV1="ABC" xterm

Then you get a new process als terminal with xterm, where ENV1 is set and valid. Try: echo $ENV1 in the xterm

4.6. specific environment variables

xxx

$? is windows: ERRORLEVEL

xxx

4.7. Set environment variable in a called script

In UNIX and also in Linux there is a strong rule, not regarded for DOS or Windows batch files:

Environment variable are valid only in the own script, and declared with

extern VAR="value"

also in called scripts, but never back from a called script.

In DOS/Windows you can set an environment variable in a

call WinBat.bat

and this variable is available also in the caller. But not in UNIX/Linux.

But there is a possibility to get the same behavior also in Linux:

`. calledShell.sh`

The characteristic is the dot before the script name. Then the called script is expanded in its lines in the calling script, and hence set of environment variable in the called script will be effective also for the caller.

5. Set the current data and time to an environment variable

This can be often used to name a file with the current time stamp.

DATE=`date -I`

This invokes date -I and sets the output to the environment variable DATE.

DATE=`date -I'minutes'`

See the options of date, and note the different apostrophs.

  • The back tick apostrop (0x60) is used to fill an environment variable.

  • The normal apostrop (0x27) is used for the date …​ option.

6. meaning of ; and & on end of a cmd

& after a command line means simple, the command is started, runs, but the command line control goes back to the console which is used to start. That is done because the command runs in a sub shell.

But also the sub shell will be terminated if the current console is terminated, due to the "hang up" signal on OS level.

It means

nohup cmd args &

starts the command really independently. see next chapter. nohup "no hangup" - continue the started process though the terminal is closed

; after a command is used to write a next command in the same line. It is only the separator between commands in the same line. Writing ; on end of line it has no meaning and no effect. But it looks nice.

7. nohup "no hangup" - continue the started process though the terminal is closed

The nohup is very successfull to replace javaw (Windows) on Linux, it is better than javaw:

On Windows:

javaw -jar ...

Starts java without console, it means cmd control returns immediately to the cmd window, and the java process runs independent. But how outputs to console are gathered? I had realized - not at all.

On Linux javaw is not available, because Linux has better solutions.

On Linux

nohup java -jar ... 1>path/to/out 2>path/to/err.txt &

starts the java process, returns to the console (because & on end) and writes out and err to the given files. The files can also be /dev/null or stored on a RAMDISK to care with the SDD

If no output redirection is used, a file ./nohup.out is created in the current directory, which is often a non proper solution. Use redirection is anytime recommended!

8. Calling arguments of a bash script

$1 First argument

${1} Same as $1

${1-} First argument, or empty if unset

${1-default} First argument, or default if unset

${1:-default} First argument, or default if unset or empty

9. Conditional execution in bash

9.1. if statement in bash

if command; then ... else ... fi

Intrinsic the return value of the executed command is used to decide whether the then or optional else branch is executed:

  • 0 it is ok, true, then then is executed

  • not 0, then the else is executed.

    test ....

is a normal command. Hence:

if test -d "dirpath"; then echo "dirpath is a directory"; fi

test returns 0 if the test is ok.

9.2. using && and || instead if

command "arguments" || echo ERROR $?

This is one command line. The || is only executed if command returns not 0 (which means not ok, or FALSE).

command && command2

command2 is only invoked if command is ok (no error, returns 0).

9.3. Bash extended conditional expressions

Note: The content of this chapter comes from ChatGPT

Bash provides extended conditional expressions using [[ …​ ]]. They are more powerful and safer in several situations than the traditional [ …​ ] (the test command).

9.3.1. 1. String comparisons

[[ "$X" == "abc" ]]       # Equal
[[ "$X" != "abc" ]]       # Not equal
[[ -z "$X" ]]             # Empty string
[[ -n "$X" ]]             # Non-empty string

9.3.2. 2. Pattern matching

[[ "$FILE" == *.zip ]]    # Ends with .zip
[[ "$FILE" == GET-* ]]    # Starts with GET-

The right-hand pattern is not quoted, so * acts as a wildcard.

9.3.3. 3. Regular expressions

[[ "$X" =~ ^[0-9]+$ ]]    # Contains only digits

9.3.4. 4. Logical combinations

[[ "$X" == "abc" && "$Y" == "def" ]]

[[ "$X" == "abc" || "$Y" == "def" ]]

[[ ! "$X" == "abc" ]]

No need for -a or -o, which are problematic in traditional test.

9.3.5. 5. File tests

These are also available in traditional test:

[[ -f "$FILE" ]]          # Exists and is a regular file
[[ -d "$DIR" ]]           # Exists and is a directory
[[ -r "$FILE" ]]          # Readable
[[ -x "$FILE" ]]          # Executable
[[ -s "$FILE" ]]          # Exists and has nonzero size

9.3.6. Recommendation

For Bash scripts, use:

  • [[ …​ ]] for conditions.

  • = or == for string equality (both work).

  • && and || for logical combinations.

Note that [[ …​ ]] is a Bash extension and is not part of POSIX shell syntax. For scripts intended to run with /bin/sh, use [ …​ ] instead.

10. ln -s how does it really work

An 'official' documentation: link::https://linuxize.com/post/how-to-create-symbolic-links-in-linux-using-the-ln-command/[]

ln -s PATH_OF_LINK LINKFILE
  • LINKFILE is a path to the linked file maybe as only file name, maybe absolute, maybe relative to the current directory, as usual for all files.

  • PATH_OF_LINK is the content in this LINKFILE, independently from the current directory or all others. Hence: Using the LINKFILE as symbolic link: PATH_OF_LINK is the destination for the link starting on the directory where the LINKFILE is stored, if it is a relative path. PATH_OF_LINK won’t be checked if the ln -s is processed. Any String is taken to store as link.

11. ls list files and directories - details

link::https://linuxvox.com/blog/linux-ls-directories-only/[]

to list a specific directory which ends with LAST write

ls -l *LAST