Using sed Capture Groups (Linux/Mac)

This will be a short one and belongs to the TIL bin: until 2 days ago I did not even know about capture groups and how to use it. So, I knew how to replace a matching string/pattern in sed

bash-3.2$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v/swallow_speed/' 
As of today swallow_speed=23kph at STD
bash-3.2$ 

And how to replace from a given pattern all the way to the end of the line. Or starting from the beginning of the line to said pattern:

bash-3.2$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v=.*/swallow_speed=42/' 
As of today swallow_speed=42
bash-3.2$ echo "As of today swallow_v=23kph at STD" | sed -e 's/^.*swallow_v=/Can you believe that swallow_speed=/' 
Can you believe that swallow_speed=23kph at STD
bash-3.2$ 

If you are curious, the .* in the search pattern means "any character or list of characters here, be it zero or a lot of characters". The dot (.) does the any part and the asterisk (*) the how many. I think this is from regex, but don't quote me on that. Fine, what if I want to replace everything between two patterns, but leaving the second pattern alone? Tricky. You see, replacing everything between the two patterns, inclusive is not that hard

raub@desktop:~$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v=.*k/swallow_v=25/'
As of today swallow_v=25ph at STD
raub@desktop:~$ 

But to preserve the second pattern we need to use the Capture Groups mentioned in the title of this article. And that makes sense because if it is on the title I better use it. So, we are supposed to surround the capture group pattern with parenthesis and then we can refer to them. If it does not make sense, I too was confused, so let's keep on using our test string:

raub@desktop:~$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v=.*(k)/swallow_v=25$1/'
As of today swallow_v=23kph at STD
raub@desktop:~$ 

Er, it does not seem to have worked according to the plan. In fact, it was supposed to at least grab swallow_v=23k but as you can see it did not find the pattern. Is $1 the proper way to output the captured string? Going nowhere slowly.

After much soul searching, I found the -e requires the parenthesis to be escaped. And, the capture group pattern is output using \1 instead of $1.So we try again:

raub@desktop:~$ echo "As of today swallow_v=23kph at STD" | sed -e "s/swallow_v=.*\(k\)/swallow_v=25\1/"
As of today swallow_v=25kph at STD
raub@desktop:~$ 

much better!

What about the Mac? Same thing (the bash-3.2$ you have seen all day is it; the raub@desktop:~$ is the Linux box):

bash-3.2$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v=.*\(k\)/swallow_v=25\1/' 
As of today swallow_v=25kph at STD
bash-3.2$ 

I am not going to say it is perfect though:

bash-3.2$ echo "As of today swallow_v=23kph at STD" | sed -e 's/swallow_v=.*\([[:blank:]]\)/swallow_v=25\1/' 
As of today swallow_v=25 STD
bash-3.2$ echo "As of today swallow_v=23kph at    STD" | sed -e 's/swallow_v=.*\([[:blank:]]\)/swallow_v=25\1/' 
As of today swallow_v=25 STD
bash-3.2$ 

References

• A site I think has lots of interesting sed examples.

Testing for multiple strings without much clutter using powershell

Yes, this hopefully will be quick, and yes it is powershell, which does not make me feel as dirty as if it was Windows. So hear me out.

I wrote a script a while ago that I needed to look for a pattern inside a string and then do something. In its simplest form, the code could look like this (second line is there to show the entire string):

$the_string = "There are pickles in a jar"
$the_string

if ($the_string -match "pickles")
{
        "Found me pickles"
}
else
{
        "nothing to declare"
}

If you wonder why I am using -match instead of -contains, there is a nice discussion you want to check. At least I learned a lot from it. When we run the script, which is henceforth called switchtest.ps1, we get

PS C:\Users\dalek> powershell -file .\dev\switchtest.ps1
There are pickles in a jar
Found me pickles
PS C:\Users\dalek>

So far so good. But, what if we want to do things based on whether other substrings are in the string? After all, instead of one single string we might be looping over a list of them, or reading a file and doing things based on what we find on a line-by-line-basis. We could add more if statements but that gets nasty quickly:

$the_string = "There are pickles in a jar"
$the_string

if ($the_string -match "pickles")
{
        "Found me pickles"
}

elseif ($the_string -match "there")
{
        "There is here"
}
else
{
        "nothing to declare"
}

Notes:

• By default powershell is case insensitive; this is consistent with how DOS and Windows behaves, which is exactly the opposite of UNIX in general and Linux specifically.
• If you have used other programming languages like python or C, you might remember switch statements (switch, then case-this and case-that). And, they are also implemented in powershell without the case word being explicitly used but the behavior is there for all to see.
• -match looks for the substring anywhere in the string. So, it would find there in:
  • $the_string = "There are pickles in a jar"
  • $the_string = "Are there pickles in a jar?"
  • $the_string = "Pickles are in a jar over there"

Let's now redo switchtest.ps1 using case:

$the_string = "There are pickles in a jar"
$the_string

switch -wildcard ($the_string)
{
        "*pickles*" {"Found me pickles"}
        "there*" {"There is here"}
        default {"nothing to declare"}
}

and then run it

PS C:\Users\dalek> powershell -file .\dev\switchtest.ps1
There are pickles in a jar
Found me pickles
There is here
PS C:\Users\dalek>

The -wildcard option is where the main magic is. It allows me to enter a string as the search pattern, such as "pickles". The more astute of you probably noticed the asterisk (*) surrounding the search string. Without the first, it would only look for a string that starts with "pickles"; Since we only want to look for there when it begins the string we do not need the leading *. There is a thread in the Spiceworks forum showing another example of looking for strings that begin with a, in their case, specific letter using a case statement. The second one is so it will not look only for string ending with that substring.

There is nothing forcing each case statement to be a one-liner. In fact, it would be clearer to write the second one as

// Do something if the string begins with "there" 
        "there*" 
             {
                   "There is here"
             }

specially if we are going to do more than just write out a string.

Another thing you might also have noticed is the behaviour of the output does not match the one we did using the if statement. Reason is unless we specifically tell the switch statement to stop testing once it finds a match, it will keep going down to the list (the default is only done if nothing matches. Sometimes that is exactly what you want to do, but let's assume that is not the case. Just as in other languages, the command we need to give it is break. Let's add them

$the_string = "There are pickles in a jar"
$the_string

switch -wildcard ($the_string)
{
        "*pickles*" {"Found me pickles"; break}
        "there*" {"There is here"; break}
        default {"nothing to declare"}
}

and try it once more

PS C:\Users\dalek> powershell -file .\dev\switchtest.ps1
There are pickles in a jar
Found me pickles 
PS C:\Users\dalek>

Now the output looks just like the original script, but it is much cleaner and easier to expand.

Creating a fixed length blank(spaces) string and padding others with spaces in powershell

Powershell has clever ways to create an empty string or string array that can then be increased in size. And then I need a fixed length empty string. Why? I need to create a file where each line is composed by fields which are position dependent. So I need to pad between the fields with blank spaces.

After searching for quite a while I found Ethan's reply on the bottom of thread on limiting a string to N characters. And it was so obvious I had to smack me on the head: if you want to create a string called moo that contains nothing but 24 spaces, you can do it by typing

PS C:\> $moo = " "*24
PS C:\> $moo.length
PS C:\> 24

So let's do something useful with that then!

So the plan is to write a function that given the string that will be used in a given field and the field size, it will return a properly padded string. Now, I think we should also create a second function whose only goal in life is to create a string with N blank spaces. And then the other function can call this. Reason I propose this is in case we need to provide some extra blank spaces because the format of this file might be, well, rather silly. Planning ahead is always a smart move. So, we start with our blank string generating function, which will look very similar to the $moo example above:

function BlankString($size)
{
   return " "*$size
}

If we want to test that we can modify the moo example

$moo = BlankString(24)
$moo.length

Which should return 24 as before. So far so good. Now let's create a function that will return a properly padded field. Something like

function PadString($name, $stringSize)
{
   if($name.length -lt $stringSize)
   {
      $name += BlankString ($stringSize - $name.length)
   }
   return $name
}

looks like a good starting place. Let's try it out:

$noo = PadString "My Head hurts" 48
DebugLine "[$noo]"
$noo.length

Results in

[My Head hurts                                   ]
48

If you are curious, my DebugLine function looks like this

function DebugLine($OutputMessage)
{
   Write-Host $OutputMessage -foregroundcolor red -backgroundcolor yellow
}

The [ and ] are there just so we can see where the string, blank spaces included, begins and ends. If you do not like those characters, change them to something more remarkable, say ---> and <---

Now, what if the string we are trying to put in this new string is bigger than it can handle? This would be the opposite of padding the new string: we need to chop it. So, we modify our PadString function a bit:

function PadString($name, $stringSize)
{
   if($name.length -lt $stringSize)
   {
      $name += BlankString ($stringSize - $name.length)
   }
   else
   {
      $name = $name.Substring(0, $stringSize)
   }
   return $name
}

So, if we redo the same test we did before, but this time we make our resulting string shorter,

$noo = PadString "My Head hurts" 8
DebugLine "[$noo]"
$noo.length

the result is now truncated

[My Head ]
8

And we can still create weird blank spaces using BlankString as needed to meet the file format requirments.

Uninstalling annoying packages using SCCM

It seems this week is Bad Windows Developer Week for me.

Do you remember that unnamed software package that forced me to create some crappy code to convert UTF-8 to ASCII7? Well, I do need to upgrade it. And plan on letting SCCM do most of the heavy lifting.

Now, sensible developers will make their upgrade package uninstall the old one by itself. Or maybe installed the packages using wmic so we can query them

and uninstall them

And then there are certain software installers that laugh at standards and place their packages wherever they feel like. In a previous article we talked about how to find how to uninstall those packages, so let's apply that knowledge here.

For this article we will create a windows batch file to uninstall the program. In a future article we will do the same using powershell. The command we will use to look into the registry is, surprisingly enough, reg query which needs to know the path in the registry

C:\Users\raub\dev> reg query "HKLM\software\Wow6432Node\Microsoft\Win
dows\CurrentVersion\Uninstall\Careless Client 12.3.4_r3" /v UninstallString

HKEY_LOCAL_MACHINE\software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\
Careless Client version 12.3.4 r3
    UninstallString    REG_SZ    "C:\Windows\unins000.exe"

C:\Users\raub\dev>

Here is what it looks like in a batch file:

@echo off

set _uninst_string=reg query "HKEY_LOCAL_MACHINE\software\Wow6432Node\Microsoft\
Windows\CurrentVersion\Uninstall\Careless Client version 12.3.4_r3" 
/v UninstallString

for /f "tokens=3" %%R in ('%_uninst_string% ^| find "C:"') do (set pickles=%%R)

rem pickles here is the command. To run it we might want to do something like
start /wait %pickles% /SILENT

Note: The set _uninst_string line ends at UninstallString; I just split it into 3 lines to make it easier to read it.