2016-02 on Funky Si's Blog (Dev)

Coding myself into a corner

funkysi1701@gmail.com (funkysi1701) — Thu, 25 Feb 2016 20:00:45 +0000

I spend an awful lot of my working life adding new features and improvements to a legacy database system.

A legacy system is an old method, technology, computer system, or application program, “of, relating to, or being a previous or outdated computer system.” Often a pejorative term, referencing a system as “legacy” often implies that the system is out of date or in need of replacement.

My particular legacy system is Microsoft Access. We use Microsoft Access to provide the front end for all our internal databases. The particular technology we use is ADP files which are only supported in Access 2000, 2003 or 2010.

Microsoft has already dropped support for Access 2000 and 2003, and support for 2010 is due to be dropped in 2020.

So why am I coding myself into a corner? Its simple for every feature or improvement I create in Access, I increase the amount of features I need to create or port to a new front end.

This is a very depressing thought, everything I create will need creating again. I am not saving myself work but increasing the amount of work I need to do again.

The bad news is that my employers like almost all businesses demand results and like a good employee I have been delivering them. I have been promised the mythical “when we are quiet” you can work on rebuilding the database front end. I know this will likely never happen so what are my options.

Do nothing I have warned my managers that this work needs doing and it is up to them to give me the resources I need. This is not an acceptable option. Firstly I am not future proofing the business, 2020 will be here before we know it. Also I am not developing myself as a developer, as the only experience I am getting is with Legacy technology that has expired or will do very soon. No one wants to employ someone who only has legacy experience.

Do something I need to keep delivering results and doing what is asked of me. Like any good engineer I should be multiplying my estimates by four, meaning that I have some time that can be used for looking at the bigger picture. This is a win-win option really. Employer gets a solution that is future proof, Employee gets valuable experience in up to date technology.

What do you think? Have you been tied to legacy technology? Why not leave a comment below.

Trek review: Best of Both Worlds

funkysi1701@gmail.com (funkysi1701) — Thu, 18 Feb 2016 20:00:45 +0000

Episode Title: The Best of Both Worlds
Star Trek Type: TNG
Original Air Date: 18 June 1990 & 24 September 1990
Written By: Michael Piller
Directed By: Cliff Bole

The Borg threaten the Federation, when they kidnap Captain Picard and destroy a fleet of 39 Starships. Luckily some original thinking from Commander Riker and Lieutenant Commander Shelby saves the Federation from assimilation.

Thoughts: I love this episode! It is probably my favourite episode in the whole of trek. I can’t think of a single scene which I don’t like.

It is a two parter that actually works best as a two parter. The moment that Riker orders the Enterprise to fire on the Borg is great. At the time some thought that Patrick Stewart might leave the show and be replaced by Riker/Shelby. I am very glad that didn’t happen but it is interesting to ponder what that might be like.

The Borg soundtrack in this episode is great. Ron Jones really captures the spooky threat that the Borg are. I was always disappointed that some of that didn’t continue in First Contact and when the Borg return in Voyager.

The episode has a cracking pace. From the moment the Borg first attack the Enterprise to the moment the Borg ship explodes the show doesn’t slow down.

I am Locutus… of Borg. Resistance… is futile.
Your life, as it has been… is over. From this time forward… you will service… us.

Commander Shelby is a great character that enhances the episode and introduces conflict between characters, particularly Riker.

I have heard that Riker’s thoughts about his career was a mirror to writer Michael Piller’s thoughts about leaving the show. Luckily for us he stayed.

In a future episode of DS9 Worf comments that he felt there was nothing that they couldn’t do during this time. I have to agree the tag team of him and Data is a great one. If I want rescuing it would be Data and Worf that I would want.

My good friends at Trekmate have just reviewed part one, why not have a listen?

The Best of Both Worlds along with all 900+ Star Trek episodes is now available in Episode Atlas .

My git repository is too large!

funkysi1701@gmail.com (funkysi1701) — Thu, 11 Feb 2016 20:00:45 +0000

Today I did a clone of one of my git repositories and it took ages to download. Looking into what got downloaded it was easy to see why. The .git folder was over 500Mb in size.

I know how this has happened. This repository was created in 2013 and has been used by me as a dumping ground for lots of things related to the code which never should have been committed.

Since 2013 I have learnt a lot more about coding and git so the current version of the files in git isn’t too bad. But git keeps the history of changes for every file so bad practices like this are kept.

What can I do about this? Well what does google suggest? I found this https://stevelorek.com/how-to-shrink-a-git-repository.html (link no longer exists).

It suggests ways of listing all the large files that are stored in git and a way to remove them. As I am the only person that regularly commits to this repository I see no problem with giving it a go.

I will summarise the steps here.

git clone url

Now you need all the remote branches so there is a bash script to run

#!/bin/bash
for branch in `git branch -a | grep remotes | grep -v HEAD | grep -v master`; do
git branch –track ${branch##*/} $branch
done

Another bash script then lists the top 10 large files

#!/bin/bash
#set -x
# Shows you the largest objects in your repo’s pack file.
# Written for osx.
#
# @see https://stubbisms.wordpress.com/2009/07/10/git-script-to-show-largest-pack-objects-and-trim-your-waist-line/
# @author Antony Stubbs
# set the internal field spereator to line break, so that we can iterate easily over the verify-pack output
IFS=$’\n’;
# list all objects including their size, sort by size, take top 10
objects=`git verify-pack -v .git/objects/pack/pack-*.idx | grep -v chain | sort -k3nr | head`
echo “All sizes are in kB. The pack column is the size of the object, compressed, inside the pack file.”
output=“size,pack,SHA,location”
for y in $objects do
# extract the size in bytes
size=$((`echo $y | cut -f 5 -d ‘ ‘`/1024))
# extract the compressed size in bytes
compressedSize=$((`echo $y | cut -f 6 -d ‘ ‘`/1024))
# extract the SHA
sha=`echo $y | cut -f 1 -d ‘ ‘`
# find the objects location in the repository tree
other=`git rev-list –all –objects | grep $sha`
#lineBreak=`echo -e “\n”`
output=“${output}\n${size},${compressedSize},${other}”
done
echo -e $output

After running the script you will see details of your largest files. I had *.msi and *.exe files in the mix. To remove them run the following, where filename is the path to the file that needs removing.

git filter-branch –tag-name-filter cat –index-filter ‘git rm -r –cached –ignore-unmatch filename’ –prune-empty -f — –all

To reclaim the disk space run the following commands

rm -rf .git/refs/original/
git reflog expire –expire=now –all
git gc –prune=now
git gc –aggressive –prune=now

Now push your changes back to the remote server.

git push origin –force –all
git push origin –force –tags

Now if you do a clone it will be much smaller than before and you can get back to coding much quicker, without having to wait.

SQL Transaction Log Backups

funkysi1701@gmail.com (funkysi1701) — Thu, 04 Feb 2016 20:00:45 +0000

Like many DBAs I spend a lot of time maintaining my SQL Server backups.

From SQL Server I maintain both full backups and transaction log backups. I have often restored my full backups but until recently I have never restored a transaction log backup. All backup strategy’s are only as good as the last time you tested the restore process.

So what is a transaction log backup?

A transaction log backup contains all the transaction log records generated since the last full backup and is used to allow the database to be recovered to a specific point in time (usually the time right before a disaster strikes). Since these are incremental, if you want to restore the database to a particular point in time, you need to have all the transaction log records necessary to replay database changes up to that point in time.

How to do the restore.

First right click on Databases in SQL Management Studio and select restore database. You should then get a screen similar to this.

In source click the … to allow you to select your backup files.

Now normally I have only ever selected one file here, the *.bak file. Instead select the *.bak and all the *.trn files as well. After SQL Server has chugged for a few minutes (time will depend on number of transaction files and server/disk speed etc) the restore plan section should fill up with files.

In the destination database box, type in the name of the database you want to restore. I recommend using a different name to avoid overwriting the original database, appending Test or a datetime to the name is what I usually do.

On my test server I need to untick the take tail-log backups option off the options screen before I can execute the restore.

Now you can either check the tick boxes in the restore plan section or (more fun) click the timeline button to select at what point in time you want to restore to.

You can either select the point in time with your mouse or specify the exact point in the time textbox. Alternatively you can just select the most recent point, probably the most likely option when disaster strikes.

Now that I have tried doing this on my test server I feel much more confident that when disaster does strike I can get things restored quickly and painlessly.

How often should you run transaction backups?

The answer to this question depends on how critical your data is. Until very recently I ran mine ever 15 minutes, I have increased this to every 5 minutes, but I have seen recommendations of running it every minute . The more critical your data the more often you should run them.