Showing posts with label script. Show all posts
Showing posts with label script. Show all posts

Wednesday, September 27, 2023

Debugging bash multiline commands in vi

The ancient vi (1976), and its enhanced and slightly more modern (1991) successor vim have become unnapreciated workhorses in recent years. They come with some interesting features you expect to find in modern IDEs. Case in point is they have built-in features that help debug scripts. In today's episode, let's start with following script:

cat > multiline.sh << 'EOF'
#!/bin/bash
# Test multiline commands
# 20230926

is_this_on=true

if [ is_this_on ]; then
        echo "yes"
        ls -l \ 
                --author \
                $(pwd)
fi
EOF

Realistically, it only shows 3 things:

  • if statements respond to true/false statements (the tests we feed them return that in the end of the day)
  • The --author option offerend in modern gnu ls.
iamdeving@desktop:~/dev/scripts/shell$ ./multiline.sh 
yes
total 92
-rwxrwxr-x 1 iamdeving iamdeving iamdeving 1092 Jan 23  2020 argtest.sh
drwxrwxr-x 3 iamdeving iamdeving iamdeving 4096 Jun 28  2021 ca-stuff
-rw-rw-r-- 1 iamdeving iamdeving iamdeving  168 Jan 30  2020 commandlinetest.sh
-rwxrwxr-x 1 iamdeving iamdeving iamdeving  153 Jun 16  2021 countest.sh
[...]
iamdeving@desktop:~/dev/scripts/shell$

What is the third thing the script shows? How to break a command into multiple lines, which is done adding \ to where we want to have a break. In this case, the ls statement was broken up into 3 lines for no reason whatsoever but to show \ in action:

        ls -l \ 
                --author \
                $(pwd)

There is a potential problem in the above: if you add a space after one of the \, it will break the statement because now the \ is making the space special, not the linefeed (a.ka.a \n) character. I intentionally did that and rerun the script, leading to the following error.

iamdeving@desktop:~/dev/scripts/shell$ ./multiline.sh 
yes
ls: cannot access ' ': No such file or directory
./multiline.sh: line 11: --author: command not found
iamdeving@desktop:~/dev/scripts/shell$ 

If you are using vim, and have it configured to highlight the syntax of programming languages (it also does a great job with html, Python, github markup language, Jekyll), and others) and open the file, you will notice one of the \ is red. That indicates something is fishy around it. Moving the cursor will show there is a space after the slash:

That picture is great if

  • You have setup vi to be in "helpful editing mode." But, it is not so great to show you there is an extra space there until you manually move your cursor there (I myself highlight the entire area and see if something that should not exist is hightighted). My trick is really not that helpful unless you know to look for it.
  • You do not have vision impairements; it relies on you being able to see the colours (yes you can configure that), or being able to see at all.

Can we do something for those who do not meed the above requirements? Actually, yes. vi was created in a time where all text was one colour, usually green. So, it has aids that always work no matter the language you are editing or how fancy your terminal session is. In this case, while we have that file open, type :set list. You will not see the same file with some new characters added.

#!/bin/bash$
# Test multiline commands$
# 20230926$
$
is_this_on=true$
$
if [ is_this_on ]; then$
^Iecho "yes"$
^Ils -l \ $
^I^I--author \$ 
^I^I$(pwd)$     
fi$

The ^I represents a tab while the $ stands in for a linefeed character (we are editing a file written in Linux/UNIX/MacOS here, so it does not have carriage return characters Windows likes so much). If you look at the two lines that have a \ in the end, one of them have a space between the \ and the $ characters: we found the culprit.

If you do not want to use vi another humble option is cat:

iamdeving@desktop:~/dev/scripts/shell$ cat -A multiline.sh 
#!/bin/bash$
# Test multiline commands$
# 20230926$
$
is_this_on=true$
$
if [ is_this_on ]; then$
        echo "yes"$
        ls -l \ $
                --author \$
                $(pwd)$
fi$
iamdeving@desktop:~/dev/scripts/shell$ 

The uses for this feature do not end there. You can view special characters (besides tabs) and ensure there is nothing hidden that should not be there, which may not be as easy with helpful GUI-based IDEs.

Tuesday, November 12, 2019

Capturing the output of a command sent through ssh within script without it asking to verify host key

You have noticed when you connect to a new server, if its key is not in ~/.ssh/known_hosts ssh will ask you to verify the key:

raub@desktop:~$ ssh -i $SSHKEY_FILE cc@"$server_ip"
The authenticity of host 'headless.example.com (10.0.1.160)' can't be established.
ECDSA key fingerprint is SHA256:AgwYevnTsG2m9hQLu/ROp+Rjj5At2HU0HVoGZ+5Ug58.
Are you sure you want to continue connecting (yes/no)? 

That is a bit of a drag if you want to run a script to connect to said server. Fortunately ssh has an option just for that, StrictHostKeyChecking, as mentioned in the man page:

ssh automatically maintains and checks a database containing identification for all hosts it has ever been used with. Host keys are stored in ~/.ssh/known_hosts in the user's home directory. Additionally, the file /etc/ssh/ssh_known_hosts is automatically checked for known hosts. Any new hosts are automatically added to the user's file. If a host's identification ever changes, ssh warns about this and disables password authentication to prevent server spoofing or man-in-the-middle attacks, which could otherwise be used to circumvent the encryption. The StrictHostKeyChecking option can be used to control logins to machines whose host key is not known or has changed.

Now we can rewrite the ssh command as

ssh -o "StrictHostKeyChecking no" -i $SSHKEY_FILE cc@"$server_ip"

and it will login without waiting for us to verify the key. Of course this should only be done when the balance between security and automation meets your comfort level. However, if we run the above from a script, it will connect and just stay there with the session open, not accept any commands. i.e. if we try to send a command, pwd in this case,

ssh -o "StrictHostKeyChecking no" -i $SSHKEY_FILE cc@"$server_ip"
pwd

it will never see the pwd. Only way to get to the pwd is to kill the ssh session, when it will then go to the next command but running it in the local host, not the remote one. If we want to run pwd in the host we ssh into, we need to pass it as an argument, i.e.

raub@desktop:/tmp$ pwd
/tmpraub@desktop:/tmp$ ssh -o "StrictHostKeyChecking no" -i $SSHKEY_FILE cc@"$server_ip" pwd
/home/raub
raub@desktop:/tmp$

which means it will connect, run said command, and then exit. But, how to do that from a script? The answer is to use eval or bash -c (or other available shell):

raub@desktop:/tmp$ moo="ssh -o \"StrictHostKeyChecking no\" -i $SSHKEY_FILE cc@$server_ip"
raub@desktop:/tmp$ dirlist=$(eval "$moo pwd")
raub@desktop:/tmp$ echo $dirlist
/home/raub
raub@desktop:/tmp$ dirlist2=$(sh -c "$moo pwd")
raub@desktop:/tmp$ echo $dirlist2
/home/raub
raub@desktop:/tmp$

Now, there are subtle differences between eval and bash -c. There is a good thread in stackexchange which explains them better than I can do.

Now why someone would want to do that?

How about doing some poor-man ansible, i.e. when host on the other side does not have python or has some other idiosyncrasy. Like my home switch, but that is another story...

Saturday, June 22, 2019

Bitten by double quotes while passing arguments to a bash function

So I wrote a script to setup some networks. The script is not that important for this article; in fact I do not like said script anymore and will replace it with some wholesome ansible playbooks. What matters in it is this bit of code:

So the script is called nething.sh and kinda looks like this:

#! /bin/bash
# Populate openstack networks

HEAD_NODE_IPADDRESS="10.0.0.30"
COMPUTE_NOTE=$HEAD_NODE_IP_ADDRESS
SEC_GROUP="custom-1"
NET_TYPE="flat"
PHYSICAL_NETWORK_DEFAULT="extnet"
NET_NAME="public"
SUBNET_NAME="subnet-${NET_NAME}"
CIDR="10.0.0.0/24"
DNS_NAMESERVER1="10.0.0.10"
GATEWAY_IP="10.0.0.2"
DHCP_ALLOCATION_START="10.0.0.249"
DHCP_ALLOCATION_END="10.0.0.254"

# Boring things here
function CreateNetwork
{
   net_name=$1
   net_id=$2          # vlan tag, segmentation ID
   net_type=$3
   subnet_name=$4
   subnet_range=$5    # Subnet is not necessarily as wide as its parent network
   gateway=$6
   subnet_dhcp_start=$7
   subnet_dhcp_end=$8
   nameserver=$9
   physical_net=$10

   if [[ ${net_id} == "EXT" ]]; then
      openstack network create --provider-network-type ${net_type} \
[...]
}

# Public Network
   CreateNetwork \
      $NET_NAME \
      "EXT" \
      $NET_TYPE \
      $SUBNET_NAME \
      $CIDR \
      $GATEWAY_IP \
      $DHCP_ALLOCATION_START \
      $DHCP_ALLOCATION_END \
      $DNS_NAMESERVER1 \
      $PHYSICAL_NETWORK_DEFAULT

which defines a few constants on the top and then pass them to a function in the end of the script.

Every time I ran the script, it kept crashing and burning, and I did not know why. So I ran the script as bash -x nething.sh so it would spit out each line/variable as it passed through them. Here is how the output looks like:

+ HEAD_NODE_IPADDRESS=10.0.0.30
+ COMPUTE_NOTE=
+ SEC_GROUP=custom-1
+ NET_TYPE=flat
+ PHYSICAL_NETWORK_DEFAULT=extnet
+ NET_NAME=public
+ SUBNET_NAME=public_subnet
+ CIDR=10.0.0.0/24
+ DNS_NAMESERVER1=10.0.0.10
+ GATEWAY_IP=10.0.0.2
+ DHCP_ALLOCATION_START=10.0.0.249
+ DHCP_ALLOCATION_END=10.0.0.254
+ CreateNetwork public BLANK flat public_subnet 10.0.0.0/24 ' '
+ net_name=public
+ net_id=BLANK
+ net_type=flat
+ subnet_name=public_subnet
+ subnet_range=10.0.0.0/24
+ gateway=' '
+ subnet_dhcp_start=
+ subnet_dhcp_end=
+ nameserver=
+ physical_net=public0
+ [[ BLANK == \E\X\T ]]
+ [[ BLANK == \B\L\A\N\K ]]

The line starting with CreateNetwork public BLANK means we are going to the function CreateNetwork(); what is come after is a list of the local variables inside the function which were populated by the function call. The [[ BLANK == \E\X\T ]] is how an if test looks like under the -x option, specifically

if [[ ${net_id} == "EXT" ]]; then

To save time I will spoil the fun: take a look at the gateway variable. Why is it blank? And why every other variable after it is blank? Note that the line

+ CreateNetwork public BLANK flat public_subnet 10.0.0.0/24 ' '
tells us that
  • gateway is being passed as ' ' right from the function call.
  • No other argument after gateway is being passed.

Why?

The problem is CIDR="10.0.0.0/24", specifically the double quotes. They are expanding the string 10.0.0.0/24 and interpreting the /24 as a character. The solution is to use single quotes, as in

HEAD_NODE_IPADDRESS="10.0.0.30"
COMPUTE_NOTE=$HEAD_NODE_IP_ADDRESS
SEC_GROUP="custom-1"
NET_TYPE="flat"
PHYSICAL_NETWORK_DEFAULT="extnet"
NET_NAME="public"
SUBNET_NAME="${NET_NAME}_subnet"
CIDR='10.0.0.0/24'
DNS_NAMESERVER1="10.0.0.10"
GATEWAY_IP="10.0.0.2"
DHCP_ALLOCATION_START="10.0.0.249"
DHCP_ALLOCATION_END="10.0.0.254"

If we then run that, the strings now behave as expected:

[root@stakola ~(keystone_admin)]# bash -x nething.sh
+ HEAD_NODE_IPADDRESS=10.0.0.30
+ COMPUTE_NOTE=
+ SEC_GROUP=custom-1
+ NET_TYPE=flat
+ PHYSICAL_NETWORK_DEFAULT=extnet
+ NET_NAME=public
+ SUBNET_NAME=public_subnet
+ CIDR=10.0.0.0/24
+ DNS_NAMESERVER1=10.0.0.10
+ GATEWAY_IP=10.0.0.2
+ DHCP_ALLOCATION_START=10.0.0.249
+ DHCP_ALLOCATION_END=10.0.0.254
+ CreateNetwork private 6855 vlan private_subnet 192.168.55.0/24 192.168.55.1 192.
168.55.100 192.168.55.200 10.0.0.10 physnet1
+ net_name=private
+ net_id=6855
+ net_type=vlan
+ subnet_name=private_subnet
+ subnet_range=192.168.55.0/24
+ gateway=192.168.55.1
+ subnet_dhcp_start=192.168.55.100
+ subnet_dhcp_end=192.168.55.200
+ nameserver=10.0.0.10
+ physical_net=private0
+ [[ 6855 == \E\X\T ]]
+ [[ 6855 == \B\L\A\N\K ]]
[...]

Moral of the story: know when to use single and double quotes!

Friday, November 30, 2018

VMWare ESXI Gripes: How not to set a ESXi host in maintenance mode

So I want to patch a ESXi host (it is by itself instead of part of a cluster). First thing I do is to tell it to go to maintenance mode:

[root@vmhost2:~] esxcli system maintenanceMode set --enable on


[crickets]

and Then I wait and wait and wait and wait some more. And then 35 minutes passed since I started that and nothing seems to be happening, which is why I decided to take the time while waiting and start typing this article. It does feel like it is trying to pause all the running guests in some convoluted way (more on that later). I checked the Place a Host in Maintenance Mode chapter in the official ESXi and vCenter docs, and this is what it says about this:

The host is in a state of Entering Maintenance Mode until all running virtual machines are powered down or migrated to different hosts.

Lovely. But, this is taking way too long. I mean, the script I wrote to save/suspend and resume ESXi guests is much faster than whatever the maintenance mode is doing. Maybe I should have done that first.

It is now almost an hour and it still have not finished doing whatever it is trying to do. So, I got annoyed:

[root@vmhost2:~] /etc/init.d/hostd restart
watchdog-hostd: Terminating watchdog process with PID 66942
hostd stopped.
hostd started.
[root@vmhost2:~] /etc/init.d/vpxa  restart
watchdog-vpxa: Terminating watchdog process with PID 67479
vpxa stopped.
[root@vmhost2:~] /etc/init.d/vpxa  status
vpxa is running
[root@vmhost2:~] vim-cmd /hostsvc/maintenance_mode_exit
The operation is not allowed in the current state.
[root@vmhost2:~]

See how it is giving me a hard time? I can play that game: time to unleash my script:

[root@vmhost2:/vmfs/volumes/5b7a0d7d-7f2b6678-68c8-00224d98ad4f/var/tmp] ./save
_runningvms.sh save
Suspending VM...
Suspending VM...
Suspending VM...
Suspending VM...
Suspending VM...
Suspending VM...
[root@vmhost2:/vmfs/volumes/5b7a0d7d-7f2b6678-68c8-00224d98ad4f/var/tmp]

and now I know I can safely do a Windows on this vm host:

[root@vmhost2:~] reboot
[root@vmhost2:~]

And now the vms are saved, I have no problem whatsoever putting this host in maintenance more. In fact, doing

[root@vmhost2:~] esxcli system maintenanceMode set --enable on
[root@vmhost2:~] 

took longer to type than to run it.

Moral of the Story

Save/quit/whatever your guests before setting maintenance mode. At least if you do not have a way to move the guests to the other nodes (if you have any), like vmotion.

Monday, September 24, 2018

Using ldapsearch and ldapmodify to talk to Active Directory

Why?

Great question! Here are a few lame excuses I was able to come with:

  • I like to use command line. This is a lame excuse because Windows have powershell. But,
  • I am more comfortable with Linux than Windows. Lame excuse since
    1. How many posts in this very blog I have made about using Windows?
    2. How many of said posts I have used the GUI when I could take care of business with Powershell?
    3. What is wrong with Powershell, at least of the applications I have used it for here so far?
I do have a couple of not so lame ones though:
  • I like to be able to access the network resources from any machine in the network running any OS. If I have a Linux box in an Active Directory-controlled network, chances are I will need to authenticate the Linux box against Active Directory (AD so I can save some keytaps). AD is Kerberos + ldap + sprinkles, so I better be able to use the usual kerberos/ldap Linux tools as one day I will need to figure out why things are boink.
  • It feels like I get more info using ldapsearch than the Windows tools, which is good when I do not know the name of an attribute, or how many instaces of said attribute are in use.

Using ldapsearch

Before we go mindlessly typing things, we need some data.
We need to know the name of the ldapserver.
Yes, if you have it configured in your ldap.conf file, you should not need it. But I prefer to assume nothing. If the domain was setup properly, we can ask it directly by typing nslookup -type=srv _ldap._tcp.DOMAIN where DOMAIN is the Active Directory domain name, not the DNS one; that caught me off guard. So, if our DNS dmain is example.com and the AD domain (we are very original) is ad.example.com, we have
raub@desktop:/tmp$ nslookup -type=srv _ldap._tcp.ad.example.com
Server:         192.168.0.10
Address:        192.168.0.10#53

Non-authoritative answer:
_ldap._tcp.ad.example.com   service = 0 100 389 ADDC0.ad.example.com.
_ldap._tcp.ad.example.com   service = 0 100 389 ADDC2.ad.example.com.
_ldap._tcp.ad.example.com   service = 0 100 389 ADDC1.ad.example.com.

Authoritative answers can be found from:
ad.example.com      nameserver = addc1.ad.example.com.
ad.example.com      nameserver = ns.example.com.
ad.example.com      nameserver = ns2.example.com.
ad.example.com      nameserver = addc0.ad.example.com.
ad.example.com      nameserver = addc2.ad.example.com.
ADDC0.ad.example.com        internet address = 192.168.1.100
ADDC1.ad.example.com        internet address = 192.168.1.102
ADDC2.ad.example.com        internet address = 192.168.1.101
ns.example.com      internet address = 192.168.0.10
ns2.example.com     internet address = 192.168.0.10

raub@desktop:/tmp$
and we can use any of the ADDCN.ad.example.com (where N=0,1,2). Notice in my setup, just using ad.example.com also worked. I found out by using netcat to see if port 636 was open (I will leave the answer for "why port 636?" as an exercise to the reader)
raub@desktop:~$ nc -v ad.example.com 636
Connection to ad.example.com 636 port [tcp/ldaps] succeeded!
^C
raub@desktop:~$ 
We need to be able to authenticate against AD somehow.
For this discussion I will be using a username and password; we can also do it using a Kerberos TGT ticket.

Fun Fact: I have a user account, my normal one, which can look into some things in LDAP/AD but then I have another ("admin") account I can see more and edit stuff in AD. You will see later on me forgetting completely about that and how it affects me. But we are getting ahead of ourselves.

With that taken care of, we are going to begin by looking for some user: me

raub@desktop:~$ ldapsearch -H "ldaps://addc0.ad.example.com:636" 
-D "raub@ad.example.com" -W -b "dc=ad,dc=example,dc=com" -LLL -s sub "(CN=raub)" 
Enter LDAP Password:
dn: CN=raub,OU=Users,OU=Identity,DC=ad,DC=example,DC=com
objectClass: top
objectClass: posixAccount
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: Wrong Droid
sn: Droid
title: Entropy Creators
description: Orthodontics
givenName: Wrong
initials: B 
distinguishedName: CN=raub,OU=Users,OU=Identity,DC=ad,DC=example,DC=com
instanceType: 4
whenCreated: 20080109142820.0Z
whenChanged: 20180905201157.0Z
displayName: Droid, Wrong
uSNCreated: 243336
memberOf: CN=cookie_recipes,OU=Distribution Groups,OU=Special Users,DC=ad,DC=example,DC=com
memberOf: CN=servers,OU=Groups,OU=APE,OU=EXAMPLE,DC=ad,DC=example,DC=com
memberOf: CN=third_floor_printers,OU=Groups,OU=APE,OU=EXAMPLE,DC=ad,DC=example,DC=com
[...]
proxyAddresses: sip:raub@ad.example.com
proxyAddresses: smtp:raub@ad.example.com
proxyAddresses: X500:/o=UNC Exchange/ou=Exchange Administrative Group (FYDIBOH
 F23SPDLT)/cn=Recipients/cn=raub
proxyAddresses: SMTP:raub@email.example.com
displayNamePrintable: Wrong Droid
name: Wrong Droid
[...]
sExchPoliciesExcluded: {26491cfc-9e50-4857-861b-0cb8df22b5d7}
msExchUserAccountControl: 0
msExchELCMailboxFlags: 2
msRTCSIP-PrimaryHomeServer:
[...]
msExchOWAPolicy: CN=Default,CN=OWA Mailbox Policies,CN=Exchange,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=ad,DC=example,DC=com

# refldaps://ForestDnsZones.ad.example.com/DC=ForestDnsZones,DC=ad,DC=example,DC=com

# refldaps://DomainDnsZones.ad.example.com/DC=DomainDnsZones,DC=ad,DC=example,DC=com

# refldaps://ad.example.com/CN=Configuration,DC=ad,DC=example,DC=com

raub@desktop:~$

That is probably more info than you wanted to know about someone, but this has its applications. Since we now know every attribute associated with a given user (I should not be that special, at least as far as AD is concerned), we can build customized queries looking for only a specific bit of info. For instance, let's just get the groups I belong to or am a member of (hint hint):

raub@desktop:~$ ldapsearch -H "ldaps://addc0.ad.example.com:636" 
-D "raub@ad.example.com" -W -b "dc=ad,dc=example,dc=com" -LLL -s sub "(CN=raub)" memberOf 
dn: CN=raub,OU=Users,OU=Identity,DC=ad,DC=example,DC=com
memberOf: CN=cookie_recipes,OU=Distribution Groups,OU=Special Users,DC=ad,DC=example,DC=com
memberOf: CN=servers,OU=Groups,OU=APE,OU=EXAMPLE,DC=ad,DC=example,DC=com
memberOf: CN=third_floor_printers,OU=Groups,OU=APE,OU=EXAMPLE,DC=ad,DC=example,DC=com

Fancy, huh? Now, if instead of doing (CN=raub) we did (CN=*raub*), it would return every entry that has a CN with raub in it. In my case that would mean two entries, raub and raub.admin (if you remember our Fun Fact you will know about it), but it could also have returned a device whose name matches that. And, they would have been printed one after the other (I do wonder if the order depends on the order they were added to LDAP/AD).

Using ldapmodify

Ok, we established we can probulate Active Directory using common household Linux/UNIX query tools. What if we want to change something? Let's say we have an AD group (specifically a distribution list) called moustache_operators (for those who own and operate moustaches) and want to add a member and delete another.

Why I do like ldapmodify to edit LDAP/AD

Main reason is because I can create a file (in the LDIF format) at my leisure (i.e. think about what I want to do) with pretty commands and comments describing what I want to do. If I like what I did, I can then document it and maybe even save the file in a wiki or somewhere that can be fed to Ansible/Puppet/Chef/Docker and reused.

Our little LDIF file, let's call it change.ldif, could look like this:

# Let's define the entity we will be fiddling with
dn: CN=moustache_operators,OU=Distribution Lists,OU=Special Users,DC=ad,DC=example,DC=com
# And then what we will be doing with it
changetype: modify
delete: member
member: CN=baldone,OU=Users,OU=Identity,DC=ad,DC=example,DC=com
# Separator because we will be doing another change
-
add: member
member: CN=raub,OU=Users,OU=Identity,DC=ad,DC=example,DC=com

So let's try it:

raub@desktop:~$ ldapmodify -H "ldaps://addc0.ad.example.com:636" -D "raub@ad.example.com" -x -W -f change.ldif
Enter LDAP Password:
modifying entry "CN=moustache_operators,OU=Distribution Lists,OU=Special Users,DC=ad,DC=example,DC=com"
ldap_modify: Insufficient access (50)
        additional info: 00002098: SecErr: DSID-03150F93, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0

raub@desktop:~$ 

Why is it not working? Well, do you remember the Fun Fact I mentioned earlier in this article? This is how it shows it's ugly head. I should have used my raub.admin@ad.example.com account instead of raub@ad.example.com. If we do it right, it then works. I will not show the output of a successful connection here; what matters is verifying the deed is done, and we can do it using dear ol' ldapsearch:

raub@desktop:~$ ldapsearch -H "ldaps://addc0.ad.example.com:636" 
-D "raub@ad.example.com" -W -b "dc=ad,dc=example,dc=com" -LLL -s sub "(CN=raub) memberOf" dn: CN=raub,OU=Users,OU=Identity,DC=ad,DC=example,DC=com
memberOf: CN=cookie_recipes,OU=Distribution Groups,OU=Special Users,DC=ad,DC=example,DC=com
memberOf: CN=servers,OU=Groups,OU=APE,OU=EXAMPLE,DC=ad,DC=example,DC=com
memberOf: CN=third_floor_printers,OU=Groups,OU=APE,OU=EXAMPLE,DC=ad,DC=example,DC=com
memberOf: CN=moustache_operators,OU=Distribution Lists,OU=Special Users,DC=ad,DC=example,DC=com

What about Mac/OSX?

They too have ldapsearch; just use the terminal and off you go.

Monday, February 26, 2018

Testing for multiple strings without much clutter using powershell

Yes, this hopefully will be quick, and yes it is powershell, which does not make me feel as dirty as if it was Windows. So hear me out.

I wrote a script a while ago that I needed to look for a pattern inside a string and then do something. In its simplest form, the code could look like this (second line is there to show the entire string):

$the_string = "There are pickles in a jar"
$the_string

if ($the_string -match "pickles")
{
        "Found me pickles"
}
else
{
        "nothing to declare"
}

If you wonder why I am using -match instead of -contains, there is a nice discussion you want to check. At least I learned a lot from it. When we run the script, which is henceforth called switchtest.ps1, we get

PS C:\Users\dalek> powershell -file .\dev\switchtest.ps1
There are pickles in a jar
Found me pickles
PS C:\Users\dalek>

So far so good. But, what if we want to do things based on whether other substrings are in the string? After all, instead of one single string we might be looping over a list of them, or reading a file and doing things based on what we find on a line-by-line-basis. We could add more if statements but that gets nasty quickly:

$the_string = "There are pickles in a jar"
$the_string

if ($the_string -match "pickles")
{
        "Found me pickles"
}

elseif ($the_string -match "there")
{
        "There is here"
}
else
{
        "nothing to declare"
}

Notes:

  • By default powershell is case insensitive; this is consistent with how DOS and Windows behaves, which is exactly the opposite of UNIX in general and Linux specifically.
  • If you have used other programming languages like python or C, you might remember switch statements (switch, then case-this and case-that). And, they are also implemented in powershell without the case word being explicitly used but the behavior is there for all to see.
  • -match looks for the substring anywhere in the string. So, it would find there in:
    • $the_string = "There are pickles in a jar"
    • $the_string = "Are there pickles in a jar?"
    • $the_string = "Pickles are in a jar over there"
Let's now redo switchtest.ps1 using case:
$the_string = "There are pickles in a jar"
$the_string

switch -wildcard ($the_string)
{
        "*pickles*" {"Found me pickles"}
        "there*" {"There is here"}
        default {"nothing to declare"}
}

and then run it

PS C:\Users\dalek> powershell -file .\dev\switchtest.ps1
There are pickles in a jar
Found me pickles
There is here
PS C:\Users\dalek>

The -wildcard option is where the main magic is. It allows me to enter a string as the search pattern, such as "pickles". The more astute of you probably noticed the asterisk (*) surrounding the search string. Without the first, it would only look for a string that starts with "pickles"; Since we only want to look for there when it begins the string we do not need the leading *. There is a thread in the Spiceworks forum showing another example of looking for strings that begin with a, in their case, specific letter using a case statement. The second one is so it will not look only for string ending with that substring.

There is nothing forcing each case statement to be a one-liner. In fact, it would be clearer to write the second one as

// Do something if the string begins with "there" 
        "there*" 
             {
                   "There is here"
             }

specially if we are going to do more than just write out a string.

Another thing you might also have noticed is the behaviour of the output does not match the one we did using the if statement. Reason is unless we specifically tell the switch statement to stop testing once it finds a match, it will keep going down to the list (the default is only done if nothing matches. Sometimes that is exactly what you want to do, but let's assume that is not the case. Just as in other languages, the command we need to give it is break. Let's add them

$the_string = "There are pickles in a jar"
$the_string

switch -wildcard ($the_string)
{
        "*pickles*" {"Found me pickles"; break}
        "there*" {"There is here"; break}
        default {"nothing to declare"}
}

and try it once more

PS C:\Users\dalek> powershell -file .\dev\switchtest.ps1
There are pickles in a jar
Found me pickles 
PS C:\Users\dalek>

Now the output looks just like the original script, but it is much cleaner and easier to expand.

Tuesday, August 15, 2017

Connecting to multiple VPNs using one single Cisco AnyConnect

Like many here, I remote into networks to work. I access organization X's network using Cisco's AnyConnect VPN client because that is what they use. When I first got involved, they told me to login to a given url in their webserver and get the client for my machine (a MacBook Air if you are curious; I do need to get a new Linux laptop but the Mac has been working great so far). Probably if my machine machine was a company-owned laptop they would have pushed the packaged using SCCM/Chocolatey (Windows) or Casper(now called jamf)/Munki (Mac). Or ansible, but that is another bag of cats. In any case, the point is I got their package, which was configured to work on their VPN. And, it works: double-click on the silly link, connect, enter my authentication info, and off I go.

Now also need to access organization B's machines. And they also chose to use AnyConnect. And just like X they also told me to install their package. Thing is if I do that it will wipe the X configuration, which would get annoying very quickly. I did try seeing if there was a way to add another profile from the client's menu but not luck. Maybe each company disabled the option so you can only use it to access their network; I do not know. Now what I could do since this is a Mac is rename Company X's VPN folder to, say, Cisco.Old (the default folder name is Cisco and then install Company B's VPN package.

This way, if I need to go to X, I would open Cisco.Old and then run that vpn client. If I then wanted to go to B, I would quit the client, go to Cisco, and then run that client. I do not know about you, but that looks a bit cumbersome to me. And, if my laptop was running Windows, I think it would not let me install 2 instances of the client that easily. There has to be a better way.

Probulating

First of all, let's assume there is a configuration file somewhere for the AnyConnect VPN client. Since I am using OSX, chances are it has some plist-sounding name. And I found something called com.cisco.Cisco-AnyConnect-Secure-Mobility-Client.plist in my preferences folder, /Users/raub/Library/Preferences, but it does not look particularly legible from the command line (yes, I know there is probably an app to do that but I like to do things from the command line):

bplist00Ñ^A^B]UILogLocation¥^C^D^E^F^G_^PA/Users/raub/.cisco/vpn/log/UIHistory_2017.08.28.23.35.34.010.txt_^PA/Users/dalek/.cisco/vpn/log/UIHistory_2017.08.28.23.53.04.734.txt_^PA/Users/raub/.cisco/vpn/log/UIHistory_2017.08.29.00.10.34.504.txt_^PA/Users/raub/.cisco/vpn/log/UIHistory_2017.08.29.00.28.05.785.txt_^PA/Users/raub/.cisco/vpn/log/UIHistory_2017.10.04.04.44.45.284.txt^@^H^@^K^@^Y^@^_^@c^@§^@ë^A/^@^@^@^@^@^@^B^A^@^@^@^@^@^@^@^H^@^@^@^@^@^@^@^@^@^@^@^@^@^@^As

So we make a copy of it and then run

plutil -convert xml1 com.cisco.Cisco-AnyConnect-Secure-Mobility-Client.plist
to convert it to something more legible, and then look inside it:

boris:~ raub$ cat com.cisco.Cisco-AnyConnect-Secure-Mobility-Client.plist




 UILogLocation
 
  /Users/raub/.cisco/vpn/log/UIHistory_2016.12.12.13.41.31.883.txt
  /Users/raub/.cisco/vpn/log/UIHistory_2016.12.12.13.58.40.264.txt
  /Users/raub/.cisco/vpn/log/UIHistory_2016.12.12.14.15.56.295.txt
  /Users/raub/.cisco/vpn/log/UIHistory_2017.02.14.06.03.40.692.txt
  /Users/raub/.cisco/vpn/log/UIHistory_2017.07.24.21.43.25.742.txt
 


boris:~ raub$

Hmmm, that does not look like what I want. Maybe the AnyConnect client has a global configuration file somewhere. And it does, and it is called glvpn-anyconnect-profile.xml and is located in /opt/cisco/anyconnect/profile/:

boris:~ raub$ ls /opt/cisco/anyconnect/profile/
AnyConnectProfile.xsd  glvpn-anyconnect-profile.xml
boris:~ raub$

If we look into it, this xml file starts as expected with some system-wide config settings

cat /opt/cisco/anyconnect/profile/glvpn-anyconnect-profile.xml
<?xml version="1.0" encoding="UTF-8"?>
<AnyConnectProfile xmlns="http://schemas.xmlsoap.org/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://schemas.xmlsoap.org/encoding/ AnyConnectProfile.xsd">
        <ClientInitialization>
                <UseStartBeforeLogon UserControllable="true">false</UseStartBeforeLogon>
                <AutomaticCertSelection UserControllable="true">true</AutomaticCertSelection>
                <ShowPreConnectMessage>false</ShowPreConnectMessage>
                <CertificateStore>All</CertificateStore>
                <CertificateStoreOverride>false</CertificateStoreOverride>
                <ProxySettings>Native</ProxySettings>
                <AllowLocalProxyConnections>true</AllowLocalProxyConnections>
                <AuthenticationTimeout>60</AuthenticationTimeout>
                <AutoConnectOnStart UserControllable="true">false</AutoConnectOnStart>
                <MinimizeOnConnect UserControllable="true">true</MinimizeOnConnect>
                <LocalLanAccess UserControllable="true">true</LocalLanAccess>
                <ClearSmartcardPin UserControllable="true">true</ClearSmartcardPin>
                <IPProtocolSupport>IPv4,IPv6</IPProtocolSupport>
                <AutoReconnect UserControllable="true">true

But then get to the part we have been anxiously waiting for: how to access company X's vpn:

<ServerList>
                <HostEntry>
                        <HostName>Company X VPN</HostName>
                        <HostAddress>vpn.companyx.com</HostAddress>
                </HostEntry>
        </ServerList>
</AnyConnectProfile>

It does not look very complicated to me: we probably could just add a new HostEntry for Company B, as in

<ServerList>
                <HostEntry>
                        <HostName>Company X VPN</HostName>
                        <HostAddress>vpn.companyx.com</HostAddress>
                </HostEntry>
                <HostEntry>
                        <HostName>Company B VPN</HostName>
                        <HostAddress>vpn.b-company.com</HostAddress>
                </HostEntry>
        </ServerList>
</AnyConnectProfile>

and be done. And that will work. But, I think we can do one better; can we avoid cluttering the profile file? Long story short is yes. Just put something like this

cat > B-profile.xml << 'EOF'
<?xml version="1.0" encoding="UTF-8"?>
<AnyConnectProfile xmlns="http://schemas.xmlsoap.org/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://schemas.xmlsoap.org/encoding/AnyConnectProfile.xsd">
    <!--
        This section contains the list of hosts the user will be able to
        select from.
      -->
    <ServerList>
        <!--
            This is the data needed to attempt a connection to a specific
            host.
          -->
        <HostEntry>
            <!--
                Can be an alias used to refer to the host or an  FQDN or
                IP address.  If an FQDN or IP address is used, a
                HostAddress is not required.
              -->
            <HostName>Company B VPN</HostName>
            <HostAddress>vpn.b-company.com</HostAddress>
        </HostEntry>
    </ServerList>
</AnyConnectProfile>

in /opt/cisco/anyconnect/profile/:

boris:~ raub$ ls /opt/cisco/anyconnect/profile/
AnyConnectProfile.xsd  glvpn-anyconnect-profile.xml
B-profile.xml
boris:~ raub$

Now when we run the client, we can select either company's VPN:

What about Windows

I've never tried but there is a file called (starting at your homedir) .\AppData\Local\Cisco\Cisco AnyConnect Secure Mobility Client\preferences.xml which would be my starting point. The global profile folder is c:\ProgramData\Cisco\Cisco AnyConnect Secure Mobility Client\Profile.

Final thoughts

I do not like that I have to configure the different profiles at the global level; I might share this laptop with other people and would like to have my profiles uncluttered away from theirs. But, at least now I can use multiple profiles to access different networks. Looking at the Windows configuration file, I wonder if I can do that int he Mac too. That will be the subject for another article.

Monday, July 31, 2017

Downloading a single file from github (using ansible perhaps)

I was going to setup Ansible to talk to one of my Windows servers. According to the Ansible page on Windows, the easiest way to install the windows side is to download the powershell script ConfigureRemotingForAnsible.ps1 and blindly run it on the windows server. It supposedly does all the magic to install and set things up. Should I run it? I don't know, but let's try to get it. We can inspect its code once we have the file.

NOTE: Setting up Ansible on windows is not the topic of this thread. All I want to show is a way to download a single file from a github repo.

Since I do not know how to do it, let's do some searching. And then we try each method out and see what's what.

Attempt I

There is a thread in stackoverflow called How to pull a single file from a server repository in Git? which
suggested using the git clone command as in
git clone https://github.com/igniterealtime/Openfire.git \
Openfire/src/java/org/apache/mina/management/MINAStatCollector.java

Let's try it out:

raub@desktop:/tmp/rmoo$ git clone https://github.com/igniterealtime/Openfire.git \
Openfire/src/java/org/apache/mina/management/MINAStatCollector.java
Cloning into 'Openfire/src/java/org/apache/mina/management/MINAStatCollector.java'...
remote: Counting objects: 107450, done.
remote: Compressing objects: 100% (53/53), done.
Receiving objects:  14% (15868/107450), 61.11 MiB | 209.00 KiB/s
[...]
remote: Total 107450 (delta 32), reused 31 (delta 16), pack-reused 107380
Receiving objects: 100% (107450/107450), 802.60 MiB | 8.23 MiB/s, done.
Resolving deltas: 100% (63893/63893), done.
Checking connectivity... done.
raub@desktop:/tmp/rmoo$ ls Openfire/src/java/org/apache/mina/management/MINAStatCollector.java
build/          i18n/     webadmin/     LICENSE.txt  README.md
dbutil/         src/      webadmintld/  Makefile
documentation/  starter/  xmppserver/   pom.xml
raub@desktop:/tmp/rmoo$

Er, does that look like it grabbed the right thing? For some reason I thought the .java file was, well, a file and not a bunch of files and directories. At least I could swear writing .java files in vi so they were text files. Maybe I am wrong, so let's see if we can get the file I really want:

raub@desktop:/tmp/rmoo$ git clone https://github.com/ansible/ansible.git ansible
/blob/devel/examples/scripts/ConfigureRemotingForAnsible.ps1
Cloning into 'ansible/blob/devel/examples/scripts/ConfigureRemotingForAnsible.ps
1'...
remote: Counting objects: 236787, done.
remote: Compressing objects: 100% (66/66), done.
remote: Total 236787 (delta 33), reused 25 (delta 6), pack-reused 236712
Receiving objects: 100% (236787/236787), 73.53 MiB | 8.23 MiB/s, done.
Resolving deltas: 100% (152234/152234), done.
Checking connectivity... done.
raub@desktop:/tmp/rmoo$ ls
ansible/
raub@desktop:/tmp/rmoo$ ls ansible/blob/devel/examples/scripts/ConfigureRemotin$ForAnsible.ps1/
ansible-core-sitemap.xml  .gitattributes            RELEASES.txt
bin/                      .github/                  requirements.txt
CHANGELOG.md              .gitignore                ROADMAP.rst
CODING_GUIDELINES.md      .gitmodules               setup.py
contrib/                  hacking/                  shippable.yml
CONTRIBUTING.md           lib/                      test/
COPYING                   .mailmap                  ticket_stubs/
.coveragerc               Makefile                  tox.ini
docs/                     MANIFEST.in               VERSION
docsite_requirements.txt  MODULE_GUIDELINES.md      .yamllint
examples/                 packaging/
.git/                     README.md
raub@desktop:/tmp/rmoo$ ls ansible/blob/devel/examples/scripts/ConfigureRemotin$ForAnsible.ps1/
bin/        test/                     docsite_requirements.txt  ROADMAP.rst
contrib/    ticket_stubs/             Makefile                  setup.py
docs/       ansible-core-sitemap.xml  MANIFEST.in               shippable.yml
examples/   CHANGELOG.md              MODULE_GUIDELINES.md      tox.ini
hacking/    CODING_GUIDELINES.md      README.md                 VERSION
lib/        CONTRIBUTING.md           RELEASES.txt
packaging/  COPYING                   requirements.txt
raub@desktop:/tmp/rmoo$

I do not know about you, but that not look like what I really wanted: a single file. On a second thought, that sure looks like the root for the ansible git repo:

I don't know about you but the files and directories sure look familiar.

I guess it is time to try something else.

Attempt II

Let's try something else: in StackOverflow there is a thread called Retrieve a single file from a repository, which suggests

git clone --no-checkout --depth 1 git@github.com:foo/bar.git && cd bar && git show HEAD:path/to/file.txt

For this attempt, we will try to get the file ConfigureRemotingForAnsible.ps1 I want:

git clone --no-checkout --depth 1 https://github.com/ansible/ansible.git  && cd ansible && \
git show HEAD:examples/scripts/ConfigureRemotingForAnsible.ps1

Thing is that will just spit the file to the screen, literally:

raub@desktop:/tmp/rmoo$ git clone --no-checkout --depth 1 https://github.com/ansible/ansible.git  && \
cd ansible && git show HEAD:examples/scripts/ConfigureRemotingForAnsible.ps1
Cloning into 'ansible'...
remote: Counting objects: 5873, done.
remote: Compressing objects: 100% (4282/4282), done.
remote: Total 5873 (delta 962), reused 3652 (delta 660), pack-reused 0
Receiving objects: 100% (5873/5873), 7.13 MiB | 8.09 MiB/s, done.
Resolving deltas: 100% (962/962), done.
Checking connectivity... done.
#Requires -Version 3.0

# Configure a Windows host for remote management with Ansible
# -----------------------------------------------------------
#
# This script checks the current WinRM (PS Remoting) configuration and makes
# the necessary changes to allow Ansible to connect, authenticate and
# execute PowerShell commands.
#
[...]

We could improve that by saving the file into a file. But how? The quickest solution I can think of is to pipe it to a file:

file="ConfigureRemotingForAnsible.ps1" ; git clone --no-checkout --depth 1 https://github.com/ansible/ansible.git  && cd ansible && $(git show HEAD:examples/scripts/$file > $file)

will put $file inside the ansible dir:

raub@desktop:/tmp/rmoo/ansible$ ls
ConfigureRemotingForAnsible.ps1
raub@desktop:/tmp/rmoo/ansible$

Of course we can do better, like placing it on the original pwd and delete the (now temporary) ansible dir. Something like

file="ConfigureRemotingForAnsible.ps1" ; git clone --no-checkout --depth 1 https://github.com/ansible/ansible.git  && cd ansible && $(git show HEAD:examples/scripts/$file > ../$file) && cd .. && rm -rf ansible

should do just fine. But you do not have to believe on me; here's it in action:

raub@desktop:/tmp/rmoo$ file="ConfigureRemotingForAnsible.ps1" ; \
git clone --no-checkout --depth 1 https://github.com/ansible/ansible.git  && \
cd ansible && $(git show HEAD:examples/scripts/$file > ../$file) && cd .. && rm -rf ansible
Cloning into 'ansible'...
remote: Counting objects: 5873, done.
remote: Compressing objects: 100% (4282/4282), done.
remote: Total 5873 (delta 962), reused 3652 (delta 660), pack-reused 0
Receiving objects: 100% (5873/5873), 7.13 MiB | 758.00 KiB/s, done.
Resolving deltas: 100% (962/962), done.
Checking connectivity... done.
raub@desktop:/tmp/rmoo$ ls
ConfigureRemotingForAnsible.ps1
raub@desktop:/tmp/rmoo$

Just to show this is not an accident, let's validate it by applying it to get the java source file we tried to get earlier.

raub@desktop:/tmp/rmoo$ file="MINAStatCollector.java" ; git clone --no-checkout --depth 1 https://github.com/igniterealtime/Openfire.git  && cd Openfire && $(git show HEAD:src/java/org/apache/mina/management/$file > $file)
Cloning into 'Openfire'...
remote: Counting objects: 5291, done.
remote: Compressing objects: 100% (4433/4433), done.
remote: Total 5291 (delta 807), reused 3394 (delta 479), pack-reused 0
Receiving objects: 100% (5291/5291), 92.80 MiB | 8.24 MiB/s, done.
Resolving deltas: 100% (807/807), done.
Checking connectivity... done.
raub@desktop:/tmp/rmoo/Openfire/Openfire$ head MINAStatCollector.java
package org.apache.mina.management;

import static org.jivesoftware.openfire.spi.ConnectionManagerImpl.EXECUTOR_FILTER_NAME;

import org.apache.mina.core.service.IoService;
import org.apache.mina.core.service.IoServiceListener;
import org.apache.mina.core.session.IdleStatus;
import org.apache.mina.core.session.IoSession;
import org.apache.mina.filter.executor.ExecutorFilter;
import org.apache.mina.filter.executor.OrderedThreadPoolExecutor;
raub@desktop:/tmp/rmoo/Openfire$

I think we can improve it by making it more generic, which might be the subject of another post. Or something; I've been dragging on finishing this article for a few weeks now so I just want it gone.

Now what about something completely Ansible-ly?

Er, this Article is getting way longer than I originally planned. I will put the ansible side on another. How about that for raising your expectations and then expertly crushing them?

Monday, February 27, 2017

Create output filename based on input filename using powershell

Here's a situation that happened to me many times in Linux: let's say we create a script which expects the user to enter the input and output filenames. Now, what if the user forgets to enter the output filename? Should we bark or come up with a filename based on the input filename? This of course depends on the situation, but when I decided to create the output filename I would tack today's date to the input filename so they would be different.

But that was Linux and bash and python and this is Windows with powershell. And, yes, we could keep on writing in Bash using cygwin, but that owuld be cheating. Give the constraint of only running what came in Windows 7 and above (I am dating myself), let's see what we can do:

  1. Date. The date formats I like are (using Linux here, so focus on the output not the command)

    raub@desktop:~$ date +%F
    2017-01-30
    raub@desktop:~$ 
    and
    raub@desktop:~$ date +%Y%m%d
    20170130
    raub@desktop:~$ 

    Both write year using 4 digits followed by 2 digits for the month and two for the day. I know some people will cry and moan and demand the traditional US format, day/month/year, but the format I like makes sorting in a directory much easier as we are putting what changes the fastest on the end of the filename. But we are talking about powershell, not Bash or Bourne shell. That is true but now we know what we want to accomplish.

    To make it easier, I will pick one of the two formats -- YYYYMMDD -- and run with it; you can later modify the code to use the other one as exercise. The equivalent in powershell is:

    PS C:\Users\raub> get-date -format "yyyyMMdd"
    20170130
    PS C:\Users\raub> 

    Looks just like what we did above in Linux.

  2. Create filename if not given. We are reading the input filename into the script in some way or fashion. How we are doing it depends on the script and whether we should be passing options with or without flags to identify them. For now, we are going to be lazy and do the simplest thing possible: using param() on the beginning of the script.

    param($inputFile, $outputFile)

    If we have just one argument, it shall be the inputfile. If two, the second one is the outputfile. What if no arguments are passed? We can just send an error message and get out.

    function Usage
    {
       Write-Host "Usage:", $MyInvocation.MyCommand.Name, "inputfile [outputfile]"
       exit
    }
    
    if (!$inputfile)
    {
       Usage
    }

    The $MyInvocation.MyCommand.Name is a lazy way to for the script to get its own name by itself.

  3. Do something if there is no $outputFile. This is a variation of the same test we did to see if we had an $inputFile:

    function LazyOutputFilename($ifile)
    {
       $ofile = (Get-Item $ifile ).DirectoryName + '\' +  `
                (Get-Item $ifile ).BaseName + `
                '_' + (get-date -format "yyyyMMdd") + `
                (Get-Item $ifile ).Extension
       return $ofile
    }
    
    function GetOutputFilename($ifile, $ofile)
    {
       # $ofile cannot be $ifile
       # Create a $ofile if one was not given
       if (( [string]::IsNullOrEmpty($ofile) ) -or ( $ofile -eq $ifile ))
       {  
          $ofile = LazyOutputFilename $ifile
       }
    
       return $ofile
    }
    
    $outputFile = GetOutputFilename $inputFile $outputFile
    • In LazyOutputFilename() we are creating the output filename. We are putting it in the same directory as the input filename and then adding the formatted date right before the file extension.

    • The ( [string]::IsNullOrEmpty($ofile) ) checks is the output file, called $ofile inside this function, is empty. The reason we also wants to make sure the output file is not the input file is because we might be reading the input file a chunk at a time (line by line if text) so the script can handle large files without using up all the memory. If we are reading it line by line and then write right back to it, bad things might happen.

    • And, yes, we are overwriting the output filename if it gets changed in GetOutputFilename().

  4. Put everyone together.

    param($inputFile, $outputFile)
    
    function Usage
    {
       Write-Host "Usage:", $MyInvocation.MyCommand.Name, "inputfile [outputfile]"
       exit
    }
    
    <#
     Create output filename based on the full path of the input filename +
     today's date appended somewhere
     #>
    function LazyOutputFilename($ifile)
    {
       $ofile = (Get-Item $ifile ).DirectoryName + '\' +  `
                (Get-Item $ifile ).BaseName + `
                '_' + (get-date -format "yyyyMMdd") + `
                (Get-Item $ifile ).Extension
       return $ofile
    }
    
    function GetOutputFilename($ifile, $ofile)
    {
       # $ofile cannot be $ifile
       # Create a $ofile if one was not given
       if (( [string]::IsNullOrEmpty($ofile) ) -or ( $ofile -eq $ifile ))
       {  
          $ofile = LazyOutputFilename $ifile
       }
    
       return $ofile
    }
    
    if (!$inputfile)
    {
       Usage
    }
    
    $outputFile = GetOutputFilename $inputFile $outputFile

Wednesday, January 11, 2017

Creating extended ASCII file in bash and maybe powershell

Does anyone remember extended ASCII (as opposite to UTF-8)? If you never heard of them, we are not talking about a proper character list that supports Russian or Japanese languages. All we are dealing with here is iso-8859-1, whose table can be found here.

I have a document in that format I need to convert to something else; if this reminds you of some 8bitmime issues we talked about, well, let's just say we could have used this to create the test file. With that said, the current situation is that I wrote a script to manipulate it which is not preserving that format; that we can talk about in a future post.

Bottom line is I need to create a small test file that I can later throw it and the script output on hexdump.

My test file will have only 3 lines,

Olivenöl
Bayerstraße 22
München

Nothing fancy; just enough to use one extended ASCII character per line. Now let's try to create the little file. Just to be different, instead of starting on Linux we will do most of the attempts in OSX. Once we have a working system, we can see if it also works on Linux.

Attempt #1

How about if we do the lazy thing and jus cut-n-paste the 3 lines above into a text file we opened usng vim, notepad++, or some pico clone? Done. Now let's see how it looks like

bash-3.2$ cat /tmp/chartest 
Olivenöl
Bayerstraße 22
München
bash-3.2$ 

That looks very promising. In fact, this might end up being a very short article. Before I publish it, should we see what hexdump thinks of it?

bash-3.2$ hexdump -Cv /tmp/chartest 
00000000  4f 6c 69 76 65 6e c3 b6  6c 0d 0a 42 61 79 65 72  |Oliven..l..Bayer|
00000010  73 74 72 61 c3 9f 65 20  32 32 0d 0a 4d c3 bc 6e  |stra..e 22..M..n|
00000020  63 68 65 6e 0d 0a                                 |chen..|
00000026
bash-3.2$ 

Correct me if I am wrong but it seems each extended ASCII character is taking two characters to be represented instead of just one single character. For instance ö is being represented by two characters, 0xC3B6. That sounds more like UTF-8/Unicode/whatever (if you want to know what to look for, they all start with a 0xC3) but not extended ASCII. Also, it is using carriage return (CR, 0x0D in hexadecimal) and line feed (LF, 0x0A) characters to separate the lines. But this is very Windowsy, not OSX/Linux style, where lines are separated by the line feed (0x0A) character only.

Attempt #2

What if we paste the lines onto the terminal and use echo to write that to the test file? Well, let's make a single line test file and see what happens

bash-3.2$ echo "Olivenöl" > /tmp/chartest 
bash-3.2$ hexdump -Cv /tmp/chartest 
00000000  4f 6c 69 76 65 6e c3 b6  6c 0a                    |Oliven..l.|
0000000a
bash-3.2$ 

Still using two characters to represent ö; at least it did not add a CR. Now this is getting annoying; is there anything else we can try?

Attempt #3

It turns out there is and we do not need any extra stuff. You see, echo has this -e option that allows you to pass a character by its hexadecimal code. From the extended ASCII table we know that ö= 0xF6 and ß= 0xDF (just to pick two examples). We also know that CRFL = 0x0D0A. I know I whined about that before, but the reason is I want to be able to decide when I want to use those characters and when I do not want, as opposite to having some program or script making a choice for me.

Let's try again, this time passing the extended ASCII characters explicitly:

bash-3.2$ echo -e "Oliven\xf6l\x0d\x0aBayerstra\xdfe 22\x0d\x0aM\xfcnchen\x0d" > /tmp/chartest
bash-3.2$ 
bash-3.2$ cat /tmp/chartest 
Oliven�l
Bayerstra�e 22
M�nchen
bash-3.2$ hexdump -Cv /tmp/chartest 
00000000  4f 6c 69 76 65 6e f6 6c  0d 0a 42 61 79 65 72 73  |Oliven.l..Bayers|
00000010  74 72 61 df 65 20 32 32  0d 0a 4d fc 6e 63 68 65  |tra.e 22..M.nche|
00000020  6e 0d 0a                                          |n..|
00000023
bash-3.2$ 

That's more like it: only one character is used to represent each character in the file. Isn't it interesting when we cat the file it is replacing the extended characters with �? But, if hexdump says they are there that is good enough for me.

What about powershell?

Even though the name of this blog implied Unix, we use enough powershell we might as well see if we can do the same. But, we have to accept we start with a bit of a handicap: powershell really really wants to write UTF-8 or unicode instead of extended ascii/iso-8859-1. Let me show you what I mean by trying to create a small file with just one single word on it, Olivenöl. As we seen before, ö = 0xF6 = 246. And that should still be true in powershell; let's find out:

PS > 'Oliven' + [char]246 + 'l'
Olivenöl
PS >

Looks like we are getting somewhere, right? For our next trick, we will save that to a file (| out-file .\chartest.txt is equivalent to doing > .\chartest.txt.

PS > 'Oliven' + [char]246 + 'l' | out-file .\chartest.txt
PS > cat .\chartest.txt
Olivenöl
PS >

Hey chief! It seems to be working fine? Why are you make this huge drama about this? That is a very good question. I will let dear old hexdump do the talking:

$ hexdump -Cv chartest.txt
00000000  ff fe 4f 00 6c 00 69 00  76 00 65 00 6e 00 f6 00  |..O.l.i.v.e.n...|
00000010  6c 00 0d 00 0a 00                                 |l.....|
00000016

$

Each character is now represented by 2 characters. Smells like Unicode, right? ok, smart guy. Now just force it to save as ASCII then. Will do:

PS > 'Oliven' + [char]246 + 'l' | out-file -encoding ASCII .\chartest.txt
PS > cat .\chartest.txt
Oliven?l
PS >

And hexdump

$ hexdump -Cv chartest.txt
00000000  4f 6c 69 76 65 6e 3f 6c  0d 0a                    |Oliven?l..|
0000000a

$

It converted the characters into ?. Helpful, isn't it? The Microsoft Scripting Guys forum pretty much tells you should save file as unicode or UTF-8 and then convert it somehow. Far from me to disagree with them, at least in this article since it makes for a great cliffhanger. In a future article we will talk about how to get extended ASCII properly in powershell just like we did in bash. It will be a bit longer but doable.

Wednesday, August 31, 2016

Uninstalling annoying packages using SCCM

It seems this week is Bad Windows Developer Week for me.

Do you remember that unnamed software package that forced me to create some crappy code to convert UTF-8 to ASCII7? Well, I do need to upgrade it. And plan on letting SCCM do most of the heavy lifting.

Now, sensible developers will make their upgrade package uninstall the old one by itself. Or maybe installed the packages using wmic so we can query them

and uninstall them

And then there are certain software installers that laugh at standards and place their packages wherever they feel like. In a previous article we talked about how to find how to uninstall those packages, so let's apply that knowledge here.

For this article we will create a windows batch file to uninstall the program. In a future article we will do the same using powershell. The command we will use to look into the registry is, surprisingly enough, reg query which needs to know the path in the registry

C:\Users\raub\dev> reg query "HKLM\software\Wow6432Node\Microsoft\Win
dows\CurrentVersion\Uninstall\Careless Client 12.3.4_r3" /v UninstallString

HKEY_LOCAL_MACHINE\software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\
Careless Client version 12.3.4 r3
    UninstallString    REG_SZ    "C:\Windows\unins000.exe"

C:\Users\raub\dev>

Here is what it looks like in a batch file:

@echo off

set _uninst_string=reg query "HKEY_LOCAL_MACHINE\software\Wow6432Node\Microsoft\
Windows\CurrentVersion\Uninstall\Careless Client version 12.3.4_r3" 
/v UninstallString

for /f "tokens=3" %%R in ('%_uninst_string% ^| find "C:"') do (set pickles=%%R)

rem pickles here is the command. To run it we might want to do something like
start /wait %pickles% /SILENT
Note: The set _uninst_string line ends at UninstallString; I just split it into 3 lines to make it easier to read it.

Tuesday, August 30, 2016

Get info on installed packages using Powershell (and maybe also WMIC)

Warning: Expect this article to be rather vague

From what I understand, when you install a program in Windows using some kind of installing package, it writes some info about the program in the registry. And that allows us to go to the control panel and see the program being listed there.

You are probably waiting for me to ask the question I seem to ask a lot here: Can we do it from command line? Good question! First candidate as the command of choice is wmic; it really allows you to get (and set) a lot of info about the computer. Let's use some examples to see what we can do: we shall begin by asking it which CPU we have

C:\Users\raub\dev> wmic cpu get name
Name
Intel(R) Core(TM) i5-4570T CPU @ 2.90GHz

C:\Users\raub\dev>

Which bios version do we have?

C:\Users\raub\dev> wmic bios get version
Version
LENOVO - 14b0

C:\Users\raub\dev>
What about who made the motherboard?

C:\Users\raub\dev> wmic baseboard get manufacturer
Manufacturer
LENOVO

C:\Users\raub\dev>

From the previous command, we kinda expected this to be a Lenovo. So, let's call this verifying that we are in fact talking to a Lenovo. Now, would it tell us which motherboard is in the machine?

C:\Users\raub\dev> wmic baseboard get model
Model


C:\Users\raub\dev>

So it is hiding info from me. Bastard! Let's see what else we can find out about the motherboard then:

C:\Users\raub\dev> wmic baseboard get /?

Property get operations.
USAGE:

GET [<property list>] [<get switches>]
NOTE: <property list> ::= <property name> | <property name>,  <property list>

The following properties are available:
Property                                Type                    Operation
========                                ====                    =========
ConfigOptions                           N/A                     N/A
Depth                                   N/A                     N/A
Description                             N/A                     N/A
Height                                  N/A                     N/A
HostingBoard                            N/A                     N/A
HotSwappable                            N/A                     N/A
InstallDate                             N/A                     N/A
Manufacturer                            N/A                     N/A
Model                                   N/A                     N/A
Name                                    N/A                     N/A
OtherIdentifyingInfo                    N/A                     N/A
PartNumber                              N/A                     N/A
PoweredOn                               N/A                     N/A
Product                                 N/A                     N/A
Removable                               N/A                     N/A
Replaceable                             N/A                     N/A
RequirementsDescription                 N/A                     N/A
RequiresDaughterBoard                   N/A                     N/A
SKU                                     N/A                     N/A
SerialNumber                            N/A                     N/A
SlotLayout                              N/A                     N/A
SpecialRequirements                     N/A                     N/A
Status                                  N/A                     N/A
Tag                                     N/A                     N/A
Version                                 N/A                     N/A
Weight                                  N/A                     N/A
Width                                   N/A                     N/A

The following GET switches are available:

/VALUE                       - Return value.
/ALL(default)                - Return the data and metadata for the attribute.
/TRANSLATE:<table name>      - Translate output via values from <table name>.
/EVERY:<interval> [/REPEAT:<repeat count>] - Returns value every (X interval) seconds, If /REPEAT specified the command
is executed <repeat count> times.
/FORMAT:<format specifier>   - Keyword/XSL filename to process the XML results.

NOTE: Order of /TRANSLATE and /FORMAT switches influences the appearance of output.
Case1: If /TRANSLATE precedes /FORMAT, then translation of results will be followed by formatting.
Case2: If /TRANSLATE succeeds /FORMAT, then translation of the formatted results will be done.

PS C:\Users\mtavares\dev>

From the above, we could run

wmic baseboard get /all

to see all the properties and their current values. It does not look very pretty unless you format it (option /FORMAT), but you can see it has a lot of potential. In fact, here is a list of other interesting queries you can do in wmic.

All that is great, but what about packages since that is the subject of this article? Another very good question. Let's pick an example from the control panel and see if we can find it using wmic. The example I will use is the TightVNC, whose publisher is GlavSoft LLC:

C:\Users\raub> wmic product where "vendor like '%%glav%%'" get name
Name
TightVNC

C:\Users\raub>

Real life example

It is real life but I changed the name to protect the guilty

Let's try it with a program I am interested on. We use a program called Careless Data, which is produced by Cargo Cult Development. According to the control panel, it consists of the following packages:

  • Careless Client: The program the users run to connect to the Careless Server.
  • Careless Data: Data from Careless Server that is temporarily stored in an unencrypted flat file.
  • Careless Upgrades: The upgrade package that brought it to version 12.3.4 r3. Well, the Careless Client binary was also upgraded.
  • Careless Dependencies: Support libraries for the Careless Client. It is actually seen as two distinct packages, CarelessDependencies and CarelessDependenciesMSI

Let's see if we can find all those packages using wmic:

C:\Users\raub>  wmic product where "vendor like '%%cargo%%'" get name
Name
CarelessDependenciesMSI
CarelessData

C:\Users\raub>

Houston we have a problem: three out of five the packages are missing. What is going on here?

Enter Powershell

So I got annoyed and decided to go back to what I know more: Powershell. Short version (this article is getting long): I knew the information about installed packages was in HKLM:\Software, so I searched for a few installed packages and found some info in

  • HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall
  • HKLM:\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\U‌​ninstall

Here is an example

PS C:\Users\raub> get-itemproperty -path 'HKLM:\software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{09DA5EE2-7E46-4DC4-96F9-BFEE50D40659}' 


PSPath       : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{09DA5EE2-7E46-4DC4-96F9-BFEE50D40659}
PSParentPath : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall
PSChildName  : {09DA5EE2-7E46-4DC4-96F9-BFEE50D40659}
PSDrive      : HKLM
PSProvider   : Microsoft.PowerShell.Core\Registry
DisplayName  : Citrix Online Launcher
[...]


PS C:\Users\raub>

I know I did not show it above but one of the attributes we can get, besides PSPath and DisplayName, is UninstallString. so, I think we could write a script that would search those paths for programs and fetch for some attributes. Let's use the vendor name as the search criteria:

param($vendor)

$locations = ("software\microsoft\windows\currentversion\uninstall", "software\Wow6432Node\Microsoft\Windows\CurrentVer
sion\Uninstall")
$count = 0

foreach ($location in $locations)
{
   foreach ($obj in (get-childitem "hklm:\$location" | get-itemproperty | where {$_.publisher -match $vendor} ))
   {  
      write-host "Name: $($obj.displayname)"
      write-host "   PSChildName: $($obj.PSChildName)"
      write-host "   PSPath $($obj.PSPath)"
      write-host "   Publisher: $($obj.Publisher)"
      write-host "   Version: $($obj.DisplayVersion)"
      write-host "   BuildVersion: $($obj.Version)"
      write-host "   UninstallString: $($obj.UninstallString)"

      $global:count++
   }
}

write-host "Done"
write-host " $($count) entries found"

Now let's see what we can find about our careless program:

.\findinstalledprogram.ps1 cargo
Name: Careless Client 12.3.4
   PSChildName: Careless Client 12.3.4_r3
   PSPath Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\Careless Client 12.3.4_r3
   Publisher: CargoCultDevelopment LLC.
   Version:
   BuildVersion:
   UninstallString: "C:\Windows\unins000.exe"
Name: CarelessDependenciesMSI
   PSChildName: {45B39321-107B-4F40-A7DC-A4CB4BFC3051}
   PSPath Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{45B39321-107B-4F40-A7DC-A4CB4BFC3051}
   Publisher: CargoCultDevelopment
   Version: 1.00.0000
   BuildVersion: 16777216
   UninstallString: MsiExec.exe /I{45B39321-107B-4F40-A7DC-A4CB4BFC3051}
Name: CarelessDependencies
   PSChildName: {4509C469-6B21-4999-BE60-3449EE7B6EDF}
   PSPath Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{4509C469-6B21-4999-BE60-3449EE7B6EDF}
   Publisher: CargoCultDevelopment.com LLC
   Version: 1.00.0000
   BuildVersion: 16777216
   UninstallString: MsiExec.exe /I{4509C469-6B21-4999-BE60-3449EE7B6EDF}
Name: CarelessUpgrades
   PSChildName: {E20A19AD-27C1-4F59-AED2-A04636BD2575}
   PSPath Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{E20A19AD-27C1-4F59-AED2-A04636BD2575}
   Publisher: CargoCultDevelopment LLC.  
   Version: 12.3.4
   BuildVersion: 167772240
   UninstallString: MsiExec.exe /I{E20A19AD-27C1-4F59-AED2-A04636BD2575}
Name: CarelessData 12.3.4 r3
   PSChildName: {EA737DCD-71C9-4a06-97B5-BB2D4EE45564}_r3
   PSPath Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{EA737DCD-71C9-4a06-97B5-BB2D4EE45564}_r3
   Publisher: CargoCultDevelopment, LLC
   Version:
   BuildVersion:
   UninstallString: "C:\Program Files (x86)\CargoCultDevelopment\unins000.exe"
Done
 5 entries found

Back to WMIC and why you should not use it

Let's revisit using wmic again.

What is happening is that our syntax is using a WMI class, Win32_Product, that only displays products installed using Windows Installer. Now, we really should not be using Win32_Product because querying Win32_Product is quite dangerous because every time you run a query (wmic product is really a macro to wmic "select * from Win32_Product"), it will cause a consistency check of all the packages installed, leading to it trying to be helpful and trying to verify and repair the install. And that opens the door to some corruption. Microsoft suggests to use Win32reg_AddRemovePrograms, which may not show everything from wmic.

Bottom line, wmic is great but looking for package info is better done by Powershell.

Monday, August 29, 2016

Poor Man's Downconverting UTF-8 to ASCII7 in powershell

I will warn you this is really lousy and inelegant code. But I needed something quick. I will also warn that this was going to be focus primarily on doing the UTF-8 to ASCII7 conversion, but it went a bit sideways with me learning something new about Powershell. If you feel a bit disappointed, oh well.

I will do my best take the high road and not bitch and make snide comments about a certain software company whose product caused me to write this script. However, after rereading this article I think I failed miserably. You have been warned!

The Problem

So, I am dealing with yet a not so bright piece of software. Specifically, one that has a database whose fields for usernames, addresses, and even phone numbers and such things assume that those users are all living in the United States and have names like Bob or Sue or John or Travis (OK, maybe not that one since it has two syllables) who lived in towns like Cleveland or Tampa. This vendor developed the code in early 2000(!) hoping it would never have to deal with umlauts and accents and other characters. Probably they did that to save space since an ASCII7-only DB field is smaller than a UTF-aware one. Imagine how much money they would make their customers save on storage!

Then someone told their customers that there are lands past the dragons and the turtles, where strange people with mysterious habits and incomprehensible languages roam.

Like Sweden.

Imagine how shocked they were when now their software had to deal with names like Фёдор, Ljungström, Simón, 宮崎, and Häyhä and towns like Malmö. To address that, in 1987 a formal work group, which later became the Unicode Consortium was formed and, anfter much head-smacking, released its first version in 1991. Unfortunately, someone failed to mention that to this company. Fear not, however, for they were not the first and will not be the last to develop their code in ASCII and then realize they have to support other languages.

This is all nice and all, but we feed data to this program as text files. Which is in UTF-8 format since it contains people and places with names using non-ASCII characters. And as a result the database gets very unhappy.

The Solution

I called them to see if they had plan on supporting Unicode in the future. And they replied. Bottom line is they will not change their code to handle Unicode. So, if we are going to use this program we need to massage the input data.

As many times before in many different operating systems, if we are going to do this, it better be doable using some kind of scripting language. Being this Windows my language of choice will once again be Powershell. Before we write a single line of code, let's do something dangerously subversive and think about what we need to do.

  1. We need to detect the non-ASCII characters. Specifically we are talking about the characters that are not within the original 7bit ASCII (characters 0 to 127 in decimal, or 0x00 and 0x7F respectively in hexadecimal) table; we talked a bit before about detecting different files formats. From there we know the UTF-8 characters we are looking for are represented by a pair: "ö" has a UTF8 Hex of 0xc3b6. Do note both characters > 0x7F.

  2. Once we detect said character pair, we need to do something with them. We are going to convert them into a single or a combination of characters, and that is where it gets interesting. Take our friend ö: should it become o or oe? And the answer is it depends.

    Some countries came up with their own representation in ASCI7 of their non-ASCI7 characters. And they are not really universal. Also, if the import file uses some kind of fixed column formatting, we have to convert it to one single character. This is really not a coding problem; we can easily create a lookup table to replace the character. So I will just leave this here in the open for you to ponder deeply about. In this article we will cheat, and badly.

Let's talk about that cheating. Instead of bothering to do some proper converting, all we will do is replace the non-ASCII7 char pair with a single question mark (63 or 0x3F). This way, whoever sees the data will know we did something there and can (if they are the affected user) replace that "?" with whatever character they feel like. Talk about passing the hot potato around...

So, if we find a character whose decimal value is > 127, we can replace it with a "?" (decimal 63). Of course, being it UTF-8 the next character in the pair is also > 127. So, we need to replace the pair of characters with a single "?".

With that in mind, let's try coding that:

param($infile) 

# Convert UTF-8 document into ASCII7
function UTF2ASCII($encoded)
{
   $oldi = ''
   $converted = ''

   foreach ($line in $encoded)
   {
      # Convert each character in $line into a decimal number
      foreach($i in [int[]][char[]]$line)
      {
         # 0 <= ASCII7 <= 127
         if ($i -gt 127)
         {
            # KLUDGE: only change to "?" if previous char isn't.
            if ($oldi -ne 63)
            {
               $i = 63
            }
            else
            { continue }
         }
         $converted = $converted + [char]$i
         $oldi = $i
      }

   }

   return $converted
}

# Read in the contents of $infile and feed them to our function
UTF2ASCII(get-content $infile)

The if statements are the lousiest part of the code, specially the part that assumes if the previous char is "?" we must be dealing with the two-bit representation of the UTF-8 character; that is a rather tall assumption but I was in a hurry and could not think of an edge case. Now that I admitted my kludge, let's test it. The way you run this script is

utf2ascii.ps1 infile

which will output the modified text to the screen. If we want to save it into a file,

utf2ascii.ps1 infile > outfile

We also need a test file. I created it by gabbing a few paragraphs from the wikipedia page on the Automatgevär m/42 plus an extra line:

Olivenöl

Ag m/42, kitaip AG42, AG-42 ar Ljungman (šved. Automatgevär m/42) – švediškas savitaisis šautuvas, nuo 1942 m. iki 7-ojo dešimtmečio naudotas Švedijos armijos.

Šautuvą 1941 m. sukonstravo Erikas Eklundas iš kompanijos AB C. J. Ljungmans Verkstäder (Malmė, Švedija). Juos gaminti 1942 m. ėmė kompanija Carl Gustafs Stads Gevärsfaktori (Eskilstuna, Švedija). Švedijos armijai buvo pagaminta apie 30 000 Ag m/42. Tačiau šis šautuvas nebuvo pagrindinis Švedijos armijos šautuvas, juo buvo 6,5 mm pertaisomas šautuvas Mauser m/96.

Po kurio laiko buvo aptikti kai kurie šautuvo trūkumai (pvz., dujų vamzdelio rūdijimas), todėl 1953–1956 Švedijos armijos šautuvai Ag m/42 buvo modernizuoti ir pavadinti Ag m/42B. Modernizaciniai pakeitimai:

cat utftest.txt | wc -l (yes it is Linux, sue me) tells me that file is 7 lines long. So, let's run it and see what happens.

$ utf2ascii.ps1 infile
Oliven?l

Ag m/42, kitaip AG42, AG-42 ar Ljungman (?ved. Automatgev?r m/42) ? ?vedi?kas s
avitaisis ?autuvas, nuo 1942 m. iki 7-ojo de?imtme?io naudotas ?vedijos armijos
.

?autuv? 1941 m. sukonstravo Erikas Eklundas i? kompanijos AB C. J. Ljungmans Ve
rkst?der (Malm?, ?vedija). Juos gaminti 1942 m. ?m? kompanija Carl Gustafs Stad
s Gev?rsfaktori (Eskilstuna, ?vedija). ?vedijos armijai buvo pagaminta apie 30
000 Ag m/42. Ta?iau ?is ?autuvas nebuvo pagrindinis ?vedijos armijos ?autuvas,
juo buvo 6,5 mm pertaisomas ?autuvas Mauser m/96.

Po kurio laiko buvo aptikti kai kurie ?autuvo tr?kumai (pvz., duj? vamzdelio r?
dijimas), tod?l 1953?1956 ?vedijos armijos ?autuvai Ag m/42 buvo modernizuoti i
r pavadinti Ag m/42B. Modernizaciniai pakeitimai:
$

Looks great, right? Not quite. Problem with the above code is the following line:

UTF2ASCII(get-content $infile)
It just tries to read the entire file into memory in one sitting and then passes all that to UTF2ASCII(). That works fine with small files, but we will be filling memory/swap space and maybe even running out of them if the file is big enough. And that is a bit of shooting ourselves in the foot. Let's rewrite it a bit by creating a second function, ConvertFile(), which reads the file line by line. This way we need not to care how large the file is.

param($infile)
$count = 0

function ConvertFile($encoded)
{
   $oldi = ''
   $converted = ''

   foreach ($line in $encoded)
   {
      ConvertLine($line)
   }
}

function ConvertLine($line)
{
   foreach($i in [int[]][char[]]$line)
   {
      if ($i -gt 127)
      {
         if ($oldi -ne 63)
         {
            $i = 63
            $global:count++
         }
         else
         { continue }
      }
      $converted = $converted + [char]$i
      $oldi = $i
   }

   $converted = $converted + "`r`n"
   return $converted
}

ConvertFile(get-content $infile)
"$($count) characters were converted"

The $count global variable is just there to count how many times the script found and converted UTF-8 characters.

When we test the script with a large file, and found out we did solve the problem in the original code we mentioned above; you can try it out if you do not believe in me.

So, let's now do a few more tests.

$ utf2ascii.ps1 infile > outfile
$ cat outfile
Oliven?l

Ag m/42, kitaip AG42, AG-42 ar Ljungman (?ved. Automatgev?r m/42) ? ?vedi?kas s
avitaisis ?autuvas, nuo 1942 m. iki 7-ojo de?imtme?io naudotas ?vedijos armijos
.

?autuv? 1941 m. sukonstravo Erikas Eklundas i? kompanijos AB C. J. Ljungmans Ve
rkst?der (Malm?, ?vedija). Juos gaminti 1942 m. ?m? kompanija Carl Gustafs Stad
s Gev?rsfaktori (Eskilstuna, ?vedija). ?vedijos armijai buvo pagaminta apie 30
000 Ag m/42. Ta?iau ?is ?autuvas nebuvo pagrindinis ?vedijos armijos ?autuvas,
juo buvo 6,5 mm pertaisomas ?autuvas Mauser m/96.

Po kurio laiko buvo aptikti kai kurie ?autuvo tr?kumai (pvz., duj? vamzdelio r?
dijimas), tod?l 1953?1956 ?vedijos armijos ?autuvai Ag m/42 buvo modernizuoti i
r pavadinti Ag m/42B. Modernizaciniai pakeitimai:
35 characters were converted
$

Yeah, the fact the line stating how many characters were converted is in the outfile is a drag, but I will survive. I really want to count the number of lines though:

$ cat outfile | wc -l
16

$

Now that was unexpected to me, specially coming from a Linux background. How did my 7 lines became 16? You see, we run the script as

utf2ascii.ps1 infile > outfile

Thing is, in Powershell the > will split a long line to the width of the console window we are using. I read the explanation quite a few times and still am not satisfied. But, since we need this code to work, time to change it one more time!

param($ifile, $ofile)
$count = 0

function ConvertFile($originalFile, $convertedFile)
{
   $oldi = ''
   $converted = ''
   rm $convertedFile -ea SilentlyContinue

   foreach ($line in (get-content -path $originalFile))
   {
      ConvertLine($line) | `
      Out-File -Append -Encoding ASCII $ofile
   }
}

function ConvertLine($line)
{
   foreach($i in [int[]][char[]]$line)
   {
      if ($i -gt 127)
      {
         if ($oldi -ne 63)
         {
            $i = 63
            $global:count++
         }
         else
         { continue }
      }
      $converted = $converted + [char]$i
      $oldi = $i
   }

   return $converted
}

if ( [string]::IsNullOrEmpty($ifile) )
{
   Write-Host "Usage: powershell -ExecutionPolicy ByPass -File", `
               $MyInvocation.MyCommand.Name, "infile [outfile]"
   exit
}

# Create a $ofile if one was not given
if ( [string]::IsNullOrEmpty($ofile) )
{
   $ofile = ([io.path]::GetFileNameWithoutExtension("$ifile")) + `
            (get-date -format yyyymmdd) + `
            ([io.path]::GetExtension("$ifile"))
}

ConvertFile $ifile $ofile
"$($count) characters were converted"

Notes:

  • Instead of sending output to console, which then would need to be redirected to a file, this script reads the input and output filenames. If an outputfilename is not provided, it will create one based on that of the inputfile.
  • Corrected the issue of splitting the processed lines before saving them.
  • Gives usage message if no parameters are given
  • Outputfile is overwritten without questioning.
  • Only output to the stdout is the number of conversions

I am bored, let's run it

$ utf2ascii.ps1 infile outfile
35 characters were converted

$

Note that it now wrote the converted character count message to the screen as promised. How about the number of lines?

$ cat outfile | wc -l
7

$

Much better! Now all we need to do is decide if we want to keep the "?" or make some conversion table, which I will leave the decision up to you.

Wednesday, March 30, 2016

Joining two circles/cylinders in openscad 2: Code and Pretty pictures

In an previous we developed the equations to make a nice curve to join two circles (or cylinders). Note that can be used anywhere, including in 3D drawing. I will cheat by using a 3D modelling program that allows me to enter the equation in some kind of programmatic way, it is open source, and can be run in the usual operating systems. If you read the title of this article, you probably figured out by now we are going to use OpenSCAD. And you would be right.

Openscad differs from most CAD programs in that you do not draw interactively using a GUI. Instead you describe the objects using a programming language. Let me show what I mean using a really basic example which I stole from the manual:

module test1()
{
        cube([2,3,4]);
}

test1();

which will draw a cube of sides 2, 3, and 4 (new definition of the word "cube" we were not aware of; just roll with it). The files have a .scad extension by default, so I named this test file test1.scad. The main "entity" is a module. Inside a module you do your 2D or 3D drawings. You can have modules inside modules and modules calling other modules that are somewhere else. And then you need to finally call the module that calls everything else so something is drawn (we call it rendered). We can also define functions and variables (most of which behave like constants). Instead of keeping making new examples, which the online manual already has plenty, let's say we want to design some kind of hinge thingie that looks like this (gaze upon the magnificence of this engineering sketch):

We will develop the file that describe such 3D object as we go.

Since we are making a hinge, I will call the file hinge.scad. This is how a completely barebones version of the file looks like:

module hinge()
{
}

hinge();

Not much going on, but at least it will run. I will assume all my units are in mm. And I will say the two holes are 10mm and 15mm in diameter. And the hinge thickness is 2mm, or 20% of the smaller circle's diameter. Putting all of that in our little file, we have

/**********************************************************************
 * Let's make a stylish door hinge, shall we?
 *
 * NOTEs:
 * o Units are in mm
 **********************************************************************/
module hinge()
{
   // Known quantities
   hole1_dia = 10;
   hole2_dia = 15;
   hinge_thickness = hole1_dia*.2;
}

hinge();

The hole name comes from we are deciding how big the hole for the pins/bolts that will hold the hinge to the frame and door need to be. I am throwing some numbers here, but in real life those are known quantities (1: grab bolt. 2: measure its diameter). The hinge thickness is how much meat will be around those bolts so the hinge will not collapse like a bunch of broccoli. Of course the minimum thickness is material dependent.

With that said, here are some interesting things to note in the file so far:

  1. Comments are very similar to C/C++
  2. Statements end with a ;
  3. hinge_thickness may seem to be a variable but it is evaluated when the file is compiled, not when it is run.

How does the file look so far in openscad? Let's load it and see:

Note we have not drawn anything yet, so there is nothing to be seen. How about if we make those two circles? Because of the simple shape of the hinge, we can do the hard work in 2D and then extrude it. This is also a good time to decide the distance between the two holes; I'll say their centers are 32mm apart, which means the circles are 30 -7 -5 = 18mm apart, and that distance henceforth shall be named circle1_height. Let's update the code and tell it to dry the two circles:

/**********************************************************************
 * Let's make a stylish door hinge, shall we?
 *
 * NOTEs:
 * o Units are in mm
 **********************************************************************/        
module hinge()                                                                  {                                                                                  
   // Known quantities                                                             
   hole1_dia = 10;
   hole2_dia = 15;
   hinge_thickness = hole1_dia*.2;
   circle1_height = 30;

   module body()
   {
      // circle2 will be located at the origin
      circle( r=(hole2_dia + 2*hinge_thickness)/2 );                              
      // circle1 will be located at y = circle1_height from circle2
      translate([0, circle1_height, 0])                                               
      circle( r=(hole1_dia + 2*hinge_thickness)/2 );
   }

   body();
}

hinge();

So, the body starts with the two circles (we have not drilled the holes for the bolts yet), where circle2 is drawn on the center of the coordinate system (coordinate [0,0,0]) and circle1 is drawn by first moving to coordinate [0,circle1_height,0] (moving up on Y axis of circle1_height and then drawing it. It does feel like you are telling it to move an imaginary 3D plotter head around to do your drawings.

And here is how it looks like on the screen. Openscad is representing the 2D objects in 3D by giving them a thickness of one unit.

Making progress; now we need to connect the two to get the general shape of the hinge.

The Lazy Way

Do you remember the K.I.S.S. (Keep It Simple, St.. er Oh Brave Enterprising One)? It is really a variation of being lazy: what is the least amount of work you have to do to get the job done? For our case, what would be the simplest way to connect the two circles? Two points define a line, so if we use a point on each circle we can draw a line between them. Mirror that and we have enclosed the mess. We will need an easy point on each circle, I do not want to think too hard so here is my proposal: if we draw a line connecting the centers of the two circles, we can then draw the diameters perpendicular to this line. Where these two lines intersect each circle define two points as shown in the picture on the right, which we stole from a previous article.

The way we will create the region connecting the two circles is by creating a trapezoid and then joining it to the circles. This will make more sense in when you see it. So, let's modify the code and introduce a command called polygon().

/**********************************************************************
 * Let's make a stylish door hinge, shall we?
 *
 * NOTEs:
 * o Units are in mm
 **********************************************************************/        
module hinge()                                                                  {                                                                                  
   // Known quantities                                                             
   hole1_dia = 10;
   hole2_dia = 15;
   hinge_thickness = hole1_dia*.2;
   circle1_height = 30;

   module body()
   {
      // Create the trapezoid
      module fillerStuff()
      {
         polygon(points=[[-(hole2_dia + 2*hinge_thickness)/2,0],[-(hole1_dia + 2*hinge_thickness)/2,circle1_height], [(hole1_dia + 2*hinge_thickness)/2,circle1_height], [(hole2_dia + 2*hinge_thickness)/2,0]], paths=[[0,1,2,3]]);
      }
 
     // circle2 will be located at the origin
      circle( r=(hole2_dia + 2*hinge_thickness)/2 );                              
      // circle1 will be located at y = circle1_height from circle2
      translate([0, circle1_height, 0])                                               
      circle( r=(hole1_dia + 2*hinge_thickness)/2 );

      // Draw the trapezoid
      fillerStuff();
   }

   body();
}

hinge();

Let's see how it looks like:

Maybe a top view?

It is hard to notice from these pretty pictures, but one issue our code is that it is drawing 3 different objects on the screen instead of one. If all you want to do is draw something on the screen, this might sound pedantic. After all, it looks good so far. But we are approaching this 3D drawing business from a programming standpoint. And it is good programming practice to use objects made of smaller objects as needed. As in Java (gasp!) C++ (Ugh!) and Python (Irk!), once the object is defined, we can manipulate this single object. For instance, we can apply transformations -- resize, distort, move, etc -- to one single object instead of each of its components. And, OpenSCAD is smart enough not to draw the areas common to the parts that make this object. OpenSCAD calls a module() what we would call an object; I think by now you have noticed the module() is the main entity in this program.

Enough of this boring talk, let's see how we do it. So we need to join them using union(). And while we are at it, let's show the 3 different objects that make this union by adding a #, the debug modifier to the command used to draw the trapezoid:

/**********************************************************************
 * Let's make a stylish door hinge, shall we?
 *
 * NOTEs:
 * o Units are in mm
 **********************************************************************/        
module hinge()                                                                  {                                                                                  
   // Known quantities                                                             
   hole1_dia = 10;
   hole2_dia = 15;
   hinge_thickness = hole1_dia*.2;
   circle1_height = 30;

   module body()
   {
      // Create the trapezoid
      module fillerStuff()
      {
         polygon(points=[[-(hole2_dia + 2*hinge_thickness)/2,0],[-(hole1_dia + 2*hinge_thickness)/2,circle1_height], [(hole1_dia + 2*hinge_thickness)/2,circle1_height], [(hole2_dia + 2*hinge_thickness)/2,0]], paths=[[0,1,2,3]]);
      }
 
      union()
      {
         // circle2 will be located at the origin
         circle( r=(hole2_dia + 2*hinge_thickness)/2 );

         // circle1 will be located at y = circle1_height from circle2
         translate([0, circle1_height, 0])
         circle( r=(hole1_dia + 2*hinge_thickness)/2 );

         // Draw the trapezoid
         # fillerStuff();
      }
   }

   body();
}

hinge();

And here is the output. The view is still the same, but now we can clearly see the fillerStuff() module compared to the two circles.

Looks like everything is nice and rosy. Or rosy and yellow.

Or, are they?

Tangents anyone?

Our lazy way would have worked perfectly if both circles had the same diameter. But they are not. I know the last few pictures look very nice, but let me see if I can show you the issue by changing circle1_height to, say, 20:

You might have to click on the picture to see it, but the intersection between the line and the small circle looks very wrong because they are at different angles.

Er, it doesn't. I think we will need to crank up the resolution a bit by setting $fn=100.

/**********************************************************************
 * Let's make a stylish door hinge, shall we?
 *
 * NOTEs:
 * o Units are in mm
 **********************************************************************/
module hinge()
{
   // Known quantities
   hole1_dia = 10;
   hole2_dia = 15;
   hinge_thickness = hole1_dia*.2;
   circle1_height = 15; // Y-axis
   $fn=100; // crank up the resolution
[...]

That will make rendering slow but should smooth things out. I also made circle1_height even smaller (15), to the point the two circles touch themselves (bonus point: why do they are more than just touching each other?) and make where the circles touch the line even more dramatic:

That will only look nice if the line is tangent to the circle. Since said line touches both circles, it must be tangent to both of them. What we are looking for is the external common tangent, whose explanation and methods of obtaining are described in many locations; I used this one. Let's modify the code to calculate the points in the two circles that satisfy that and then draw the trapezoid around them; I will try later on to talk about how to obtain it as applied to OpenSCAD.

For now we need to know how to do asin(), functions.

Here is how the code looks like. I modified it a bit so the 4 points used to make the trapezoid are calculated before it is drawn.

/**********************************************************************
 * Let's make a stylish door hinge, shall we?
 *
 * NOTEs:
 * o Units are in mm
 **********************************************************************/        
module hinge()                                                                  {                                                                                  
   hole1_dia = 10; // Top circle
   hole2_dia = 15; // Bottom circle
   hinge_thickness = hole1_dia*.2;
   circle1_height = 15; // Y-axis
   $fn=100; // crank up the resolution

   // Useful functions
   function geta1(r1, r2, k) = k*r2/(r2 - r1);
   function getAlpha(opposite_side = 1, hypothenuse = 1) =
                     asin(opposite_side/hypothenuse);

   module body()
   {
      // Create the trapezoid
      module fillerStuff()
      {
         /************** Lazy
         point0 = [-(hole2_dia + 2*hinge_thickness)/2, 0];
         point1 = [-(hole1_dia + 2*hinge_thickness)/2, circle1_height];
         point2 = [(hole1_dia + 2*hinge_thickness)/2, circle1_height];
         point3 = [(hole2_dia + 2*hinge_thickness)/2,0];
          *************/

         r1 = (hole1_dia + 2*hinge_thickness)/2;
         r2 = (hole2_dia + 2*hinge_thickness)/2;
         a1 = geta1(r1, r2, circle1_height);
         alpha = getAlpha(r1, a1);
         echo("Opposite side for small triangle: ", a1);
         echo("Angle in degrees: ", alpha);
         y1 = circle1_height + r1*sin(alpha);
         y2 = r2*sin(alpha);
         x1 = r1*cos(alpha);
         x2 = r2*cos(alpha);
         echo(x1=x1, y1=y1, x2=x2, y2=y2);

         // Points that make the trapezoid
         point0 = [-x2, y2];
         point1 = [-x1, y1];
         point2 = [x1, y1];
         point3 = [x2, y2];

         polygon(points=[point0, point1, point2, point3], paths=[[0,1,2,3]]);
      }
 
      union()
      {
         // circle2 will be located at the origin
         circle( r=(hole2_dia + 2*hinge_thickness)/2 );

         // circle1 will be located at y = circle1_height from circle2
         translate([0, circle1_height, 0])
         circle( r=(hole1_dia + 2*hinge_thickness)/2 );

         // Draw the trapezoid
         # fillerStuff();
      }
   }

   body();
}

hinge();

We are also using the echo function to print out to the console some intermediate values. I used that to debug my code, which was wrong on the first trial and forced me to redraw the image so I could get the right equations. Here is how this output looks like:

Compiling design (CSG Tree generation)...
ECHO: "Opposite side for small triangle: ", 57
ECHO: "Angle in degrees: ", 7.05413
ECHO: x1 = 6.94701, y1 = 15.8596, x2 = 9.42809, y2 = 1.16667
Compiling design (CSG Products generation)...
Geometries in cache: 44
Geometry cache size in bytes: 154304
CGAL Polyhedrons in cache: 0
CGAL cache size in bytes: 0
Compiling design (CSG Products normalization)...
Compiling highlights (1 CSG Trees)...
Normalized CSG tree has 2 elements
Compile and preview finished.
Total rendering time: 0 hours, 0 minutes, 0 seconds

The ECHO statements are the output of the echo commands we inserted in the code. They are only printed to the console window. So, did our extra coding effort paid off? Here is the image with the new trapezoid nodes.

Note the transition between circles and straight lines look much nicer.

A Touch of Flair

I think we are making a lot of progress: we have a hinge that looks very smooth. But, let's not rest on our laurels just yet. I think we can do it even nicer, and get really close to the first picture in this article. And we shall do so by modifying our code to use the equations we created in a previous 3D drawing article. Note that once again we commented out the lines for the previous joining solution, tangents, so they are in the same code.

/**********************************************************************
 * Let's make a stylish door hinge, shall we?
 *
 * NOTEs:
 * o Units are in mm
 **********************************************************************/        
module hinge()                                                                  {                                                                                  
   hole1_dia = 10; // Top circle
   hole2_dia = 15; // Bottom circle
   hinge_thickness = hole1_dia*.2;
   circle1_height = 15; // Y-axis
   $fn=100; // crank up the resolution
   cutout_dia = 50;

   // Useful functions
   function geta1(r1, r2, k) = k*r2/(r2 - r1);
   function getAlpha(opposite_side = 1, hypothenuse = 1) =
                     asin(opposite_side/hypothenuse);
   // Fancy curves
   function getC2(a=1, b=1, c=1) = (pow(a,2) + pow(c,2) - pow(b,2))/(2*c);
   function getCx0(hip=1, side=1) = sqrt(pow(hip,2) - pow(side,2));

   module body()
   {
      // Create the trapezoid
      module fillerStuff()
      {
         /************** Lazy
         point0 = [-(hole2_dia + 2*hinge_thickness)/2, 0];
         point1 = [-(hole1_dia + 2*hinge_thickness)/2, circle1_height];
         point2 = [(hole1_dia + 2*hinge_thickness)/2, circle1_height];
         point3 = [(hole2_dia + 2*hinge_thickness)/2,0];
          *************/
         r1 = (hole1_dia + 2*hinge_thickness)/2;
         r2 = (hole2_dia + 2*hinge_thickness)/2;
         /************** Tangent
         a1 = geta1(r1, r2, circle1_height);
         alpha = getAlpha(r1, a1);
         echo("Opposite side for small triangle: ", a1);
         echo("Angle in degrees: ", alpha);
         y1 = circle1_height + r1*sin(alpha);
         y2 = r2*sin(alpha);
         x1 = r1*cos(alpha);
         x2 = r2*cos(alpha);
         echo(x1=x1, y1=y1, x2=x2, y2=y2);
          *************/

         a = (2*r2 + cutout_dia)/2;
         b = (2*r1 + cutout_dia)/2;
         c = circle1_height;
         c2 = getC2(a, b, c);  // Projection of a onto c (y axis)
         c1 = c - c2;   // Projection of b onto c (y axis)
         Cx0 = getCx0(a, c2);
         C0 = [Cx0, c2]; // Tangent circle center
         C1 = [-Cx0, c2]; // Tangent circle center
         // Tangent points as coordinates
         x2 = Cx0*r2/a;
         y2 = c2*r2/a;
         x1 = Cx0*r1/b;
         y1 = c - c1*r1/b;

         // Points that make the trapezoid
         point0 = [-x2, y2];
         point1 = [-x1, y1];
         point2 = [x1, y1];
         point3 = [x2, y2];

         // Draw it
         difference()
         {
            polygon(points=[point0, point1, point2, point3], paths=[[0,1,2,3]]);
            translate(C0)circle(r=cutout_dia/2, center=true);
            translate(C1)circle(r=cutout_dia/2, center=true);
         }
      }
 
      union()
      {
         // circle2 will be located at the origin
         circle( r=(hole2_dia + 2*hinge_thickness)/2 );

         // circle1 will be located at y = circle1_height from circle2
         translate([0, circle1_height, 0])
         circle( r=(hole1_dia + 2*hinge_thickness)/2 );

         // Draw the trapezoid
         # fillerStuff();
      }
   }

   body();
}

hinge();

Fun fact: If you are using a PLA filament 3D printer, having the outside sides of the trapezoid be curved will look nicer than going from circle to straight line. I was told it is because the printer needs to stop when going from curve to straight. But we can talk about printing our handiwork in another article.

You may have also noticed how progressively more complex the code has become. But that is a small price to pay for beauty. And now let's see what all that effort brings us

Not bad. What if we increase circle1_height to, say, 20?

Looks very smooth. Now we then take the # from

// Draw the trapezoid
         # fillerStuff();

Er, what about the holes for the pins/bolts?

Good catch! I forgot completely about them! So, how about if we change

body();

to

module body_hole()
   {
      union()
      {
         // circle2 will be located at the origin
         circle( r=hole2_dia/2 );

         // circle1 will be located at y = circle1_height from circle2
         translate([0, circle1_height, 0])
         circle( r=hole1_dia/2 );
      }
   }

   difference()
   {
      body();
      body_hole();
   }

Let's see how it looks like

A nice angled view perhaps?

And, finally, let's give some thickness to this hinge by using the linear_extrude() module (I am cheating and saying the hinge is circle1_height thick in the code excerpt below).

module body_hole()
   {
      union()
      {
         // circle2 will be located at the origin
         circle( r=hole2_dia/2 );

         // circle1 will be located at y = circle1_height from circle2
         translate([0, circle1_height, 0])
         circle( r=hole1_dia/2 );
      }
   }

   linear_extrude(height = circle1_height, center = false, convexity = 10)
   difference()
   {
      body();
      body_hole();
   }

And look at it

I think it looks very nice, how about you? If you agree with me, I think we succeeded into doing a quick intro to using OpenSCAD. Ok, it is neither quick nor really intro, for I rushed a bit in some of the equations. The ones for the fancy hinge were derived in a previous article, but that is not the case for the tangential solution. How about this: I will make later on a new article showing how those equations were derived. And then, we will do some 3D printing to see how our handiwork looks! Sounds like a plan?