A One Liner for Stripping Comments

Posted by Greg Laden on August 18, 2009
(8)
More »

Getting rid of those pesky comments in your all important configuration files.

This is a tip from a recent edition Linux Journal which I’m sure they don’t mind me passing on as long as I mention that for the regular Linux user, Linux Journal is the best source of relevant news, tips, and inspirational stuff.

The idea is to take all the pesky comments out of one of those configuration files. Comments are preceded with a “#” sign, and often large sections of the config file are “commented out.” If you just want to see the parts that actually do something, try this:

grep ^[^#] /etc/ntp.conf

In this case, ntp.conf is the configuration file. (Note that the “etc’ directory tends to hold your configuration files. The reason for that is because technicaly, the way something works is you “install it, then, to make it work, you know, etc. etc. etc. ” )

This works because the grep command uses the regular expression code (the funny looking stuff) to match (or not match) each line in the file that is fed to it. If the match is good, the line is spewed into “standard output” which by default is your terminal.

The first ‘^’ anchors the rest of the regular expression to the beginning of a line. The second bit, with the [brackets] is where you can put characters to match. The second ‘^’ is not a ‘beginning of line’ thingie, but rather, in the context of the brackets, signifies the logical “not.” So, this looks for lines that do not have a “#” at the beginning. Blank lines are also not matched because the ‘^’ character does not work on them. Therefore, all blank lines and all comment lines are ignore by this grep command.

If you want to send this stripped down text into a file instead of just onto the screen, so you can play around with it more, just use redirection:


grep ^[^#] /etc/ntp.conf > /etc/ntp.nocomments.conf

The ‘>’ symbol, which is obviously an arrow, points standard output to whatever it is pointing to, in this case, a filename. If that file already exists, then it will be overwritten without warning. Which is good, because we don’t need no stinking warnings.

(8)
More »

Comments

  1. #1 JH
    August 18, 2009

    I used to subscribe to ‘Linux Journal’ but eventually let my subscription lapse … twice. The writing just isn’t that good, and editing can only do so much. And I’m saying this even though I’m friends with one of the editors! :-)

    Also, there’s not enough stuff for the “general” Linux user. If you’re a hardcore geek who’s hip-deep in coding projects, it’s probably worth it. But not for me.

  2. #2 Greg Laden
    August 18, 2009

    I read it because I like to read about the hip-deep kernel and coding stuff even though I don’t do it myself, and it is less irrelevant than the equivalent (Linux Magazine) and a fraction of the price of the British Linux Format. BTW, Linux Format is great, but they do talk about a lot of software that seems to be not as readily available or used in distros here in the US, making me wonder about differences across international communities.

  3. #3 Nathan Myers
    August 18, 2009

    Could be better. 

      grep -v '^[[:space:]]*#'

    is better, as it picks up comment lines that don’t start in the first column, but might be preceded by spaces or tabs. (The “-v” means copy all lines that don’t match.)

    If you really only want lines with a comment marker in the first column, 

      grep -v '^#'

    is cleaner. Finally, it’s always a good idea to quote your grep pattern to protect against surprising shell interpretations of your pattern string.
     

  4. #4 Alex
    August 18, 2009

    Is there any benefit to this? I imagine that it would save a bit of space, but is the space saving significant, or does it make things run faster?

    It is still pretty sweet that you can do this.

  5. #5 Aaron Luchko
    August 19, 2009

    Alex,

    The advantage is when you have a heavily commented configuration file you can run that command and see only the lines that actually do something.

    On a side note

    grep ^[^#] /etc/ntp.conf

    is MUCH nicer than the line I’ve been using

    sed -e ‘s/^[ ]*#.*$//g’ -e ‘/^$/d’ /etc/ntp.conf

  6. #6 travc
    August 19, 2009

    Stripping comments is generally very useful, but not so much for looking at the config files in /etc. Those comments are normally things to help you actually understand what the commands/directives mean.

    De-commenting is really valuable when you are sending a file as input to a program or script. When you aren’t dealing with massive amounts of data, it is really nice just to keep it in a text file which you can edit and comment by hand. For example, I’m using a lot of csv files at the moment to keep track of annotations for field recordings. When I want to read that info into matlab (or R or whatever), the comments need to get stripped out.

    Actually, it gets a bit more complicated since the programs I’m using are more or less brain-dead handling csv files… So I have written short scripts to reformat the data into something that they can handle. De-commenting is just the universal first step.

    It is also the universal first step I include when writing a program or script which takes data (or configuration settings). It’s amazing how useful it is to be able to just comment out old values (instead of deleting them and forgetting what the original value was).

    BTW: Anyone know a good one-liner to remove sh style comments including anything following a ‘#’ on a line? Note: ‘Good’ means that it must also NOT strip ‘\#’.

  7. #7 travc
    August 19, 2009

    Damn, I forgot single quoted #s. Those really shouldn’t be stripped either. Probably not going to throw out my little decomment.c program anytime soon.

  8. #8 Greg Laden
    August 19, 2009

    Aaron: Yes, that is one reason why you would want to do this … these config files can be miles long with only a few working bits. This is a great way to get to that.

    But no, your sed line is beautiful. It would be nicer if you could have more characters that needed to be escaped, and maybe work in ‘cat’ somehow….