Bash/Shell Tips

31 July 2015

Remove Some Files But Not All

Shell Pattern (`glob` and `extglob`)

Basic Glob

?: one char; *: any number of chars; [..]: one of the char

REMARK: Very limited, trying to figure out the right pattern might be even harder and more time consuming than the toolchain method (see below).
extglob

Need to turn on this feature: shopt -s extglob.

!(.|.|.): anything that doesn't match patterns

Use extra tools

One example: ls | grep ... | xargs rm

Multiple Pipes

Common Grep Buffer Issue

Use --line-buffered, a GNU grep extension.

Example

adb shell 'getevent /dev/input/event4' | grep --line-buffered "0003 0019" | tee tmp.txt

Other Stdout Buffer Issue

Use complex while-read-line structure.

Example

tail -f logfile | while read line ; do echo "$line"| grep 'org.springframework'|cut -c 25- ; done

Label output lines with time stamps

NOTE: you may need to deal with buffer issues, refer to Multiple Pipes.

Example The following timestamp the distance sensor's output.

The core is the following one-line Perl. You can change time to localtime, if you want prettier time format.

perl -ne '$|=1; print "[" . time . "]" . ": $_"'

adb shell 'getevent /dev/input/event4' \
    | grep --line-buffered "0003 0019" \
    | perl -ne '$|=1; print "[" . time . "]" . ": $_"'

# relative time, the start of recording is the time zero.
adb logcat | perl -ne \
    '$|=1; $start = time unless $start; print "[" . (time - $start) . "]" . ": $_"'

Cope with Line Ending

The moral is to raise the awareness of this issue rather than the specific solution.

Problem

If the underlying shell returns strings delimited by CRLF then the usual shell commands like below will fail.

for ko in $(adb shell ls /system/lib/modules); adb pull "/system/lib/modules/$ko" ; done

As $(...) only removes Unix-style line ending, '\r' will be part of the string which leads to "does not exist" error.

Solution

Remove the '\r' part. The simplest way to do this is using `sed'.

for ko in $(adb shell ls /system/lib/modules | sed 's|\r|\n|'); do adb pull /system/lib/modules/$ko; done

You are also welcome to use utility like dos2unix.

Here string

awk '/^\/dev/ {print $1}' <<<"$(df /usr/bin/hexdump)"

# <=>

df /usr/bin/hexdump | awk '/^\/dev/ {print $1}'

http://tldp.org/LDP/abs/html/x17754.html#HERESTRINGSREF

Double Pipeline followed by a colon

Example let argc++ || :

colon means a nop operation, i.e. does nothing.
Useful for clearing return or error status.
This idiom says: ignores the error from the first part of OR statement.

∎

Comment: Github Issue