Tag Archives: sed

Fixing compile_commands.json for VS Code and Linux kernel sources

Posted on by
Reply

I have my Linux kernel sources setup like this:

-------------------
Fri Jun 14 00:35:24 [bash:5.2.15 jobs:0 error:0 time:2449]
jj5@virtuoso:/home/jj5/repo/git/git.kernel.org/torvalds-linux
$ git remote show origin
* remote origin
  Fetch URL: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
  Push  URL: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
  HEAD branch: master
  Remote branch:
    master tracked
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (local out of date)
-------------------

The scripts/clang-tools/gen_compile_commands.py runs just fine:

-------------------
Fri Jun 14 00:35:28 [bash:5.2.15 jobs:0 error:0 time:2453]
jj5@virtuoso:/home/jj5/repo/git/git.kernel.org/torvalds-linux
$ scripts/clang-tools/gen_compile_commands.py 
-------------------

But there is a problem with some unknown clang options, as you can see here:

Problems with clang commands in VS Code

So I wrote this collection of sed scripts to delete the problematic options:

-------------------
Fri Jun 14 00:11:28 [bash:5.2.15 jobs:0 error:0 time:1013]
jj5@virtuoso:/home/jj5/repo/git/git.kernel.org/torvalds-linux
$ cat jj5/fix-commands.sh 
#!/bin/bash

main() {

  set -euo pipefail;

  cd "$( dirname "$0" )";

  cd ..;

  local file="compile_commands.json";

  sed -i 's/-mpreferred-stack-boundary=3//g' "$file"
  sed -i 's/-mfunction-return=thunk-extern//g' "$file"
  sed -i 's/-mindirect-branch=thunk-extern//g' "$file"
  sed -i 's/-mindirect-branch-register//g' "$file"
  sed -i 's/-fno-allow-store-data-races//g' "$file"
  sed -i 's/-fconserve-stack//g' "$file"
  sed -i 's/-mrecord-mcount//g' "$file"

}

main "$@";
-------------------

Bug fixed! Thanks to my mates on IRC for helping me fix this one. This clangd extension for VS Code seems to work really well.

p.s. another possible solution is here.

Mail log IP address count

Posted on by
Reply

The following monster will parse the mail log and report on unique host connections along with a count.

cat /var/log/mail.log | \
  grep ' connect from unknown' | \
  awk '{ print $8 }' | \
  sort | \
  sed -n 's/.*\[\([0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\)\].*/\1/p' | \
  awk '{count[$1]++} END {for (word in count) print count[word], word}' | \
  sort -n

Removing colour code special characters with sed

Posted on by
Reply

I want to post-process the output of an ‘ls’ command with ‘sed’ so that I can remove the ‘./’ prefixes that I can’t avoid going into the ls output (this is a result of using ‘find’ safely).

The thing is, if I pipe ls output to sed, then the default –color=auto setting applies and ls detects that it’s not talking to a terminal so doesn’t output colour codes. But I want colour codes, usually, so I need to change the ls command to use –colour=always, which I’ve done. This means I can have colour and also have sed format the ls output.

The problem is then what happens if I want to pipe my output to ‘less’? Then the colour code commands appear as garbage in the output stream. So, usually I want colour codes, and sometimes I don’t.

I found this article, Remove color codes (special characters) with sed, which helped me come up with the following bash alias:

 alias noco='sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g"'

So now that I have the ‘noco’ alias (short for “no colour”) I can pipe my output through that if I want the colour codes removed, which I can apply before piping output to less.

It’s a little bit annoying that I have to do things this way but I haven’t been able to think of a better way to make it all work and this all seems to get the job done.