2015-03 on Funky Si's Blog (Dev)

Monitoring Screens

funkysi1701@gmail.com (funkysi1701) — Wed, 25 Mar 2015 20:00:45 +0000

We all know that it is important to monitor your servers and services, so you can spot issues before they become problems. I personally have spent a lot of time configuring nagios to email me about issues and I have recently been configuring various different alerts in Azure.

My old boss has this idea that I should have a big monitor screen displaying all the vital stats of my servers and services, I personally disagree with this idea and think that notifications on my phone and email alerts are sufficient. He will no doubt correct my thinking when he reads this, but I believe part of his thinking is to make the monitoring of your infrastructure move visible and make it obvious to anyone that walks past that you have your eye on everything.

For the purpose of this blog post lets assume he has convinced me and I have convinced my actual boss to spend money on the required technology to do this (No easy feat). What exactly would I display on this screen?

I have Google Chromecast that I use for streaming various things to my TV, this is a relatively cheap bit of technology that could allow a TV or monitor to display a web page with the required stats displayed.

The two main sources of information that I want to display are New Relic for monitoring my azure websites and Nagios for monitoring my internal servers. New Relic allows you to easily export live performance data as iframes so I quickly threw together a web page full of these graphs. However if you have a static screen on the wall you don’t want to have to scroll to see different information so I needed to come up with another way to display this information.

My first thought was a slide show. There are lots of javascript scripts that cycle through a series of images like a slideshow, this could be adapted to cycle through a series of iframes and display everything I want.

My script goes something like this and requires jquery as well as javascript. First of all the script waits for the page to load completely with the ready function, it then defines the urls which will be put into the iframe one at a time. It than counts the number of urls you have. It then loops through changing the contents of the src attribute in the iframe every few seconds, in my example it changes every 9 seconds but once this is used in production you may want to increase this.

<script type="text/javascript">
$(document).ready(function(){
 var locations = ["URL1", "URL2", "etc"];
 var len = locations.length;
 var iframe = $('#frame');
 var i = 0;
 setInterval(function () {
 iframe.attr('src', locations[++i % len]);
 }, 9000);
});
</script>

Now what information wants to be included in a script like this? Showing too much performance data can almost be as bad as not doing it at all as problems gets drowned out in the noise. For me I have performance of my websites, followed by Nagios problems, followed by the azure status page, followed by memory usage of all my servers and lastly showing number of connections to my databases. Another question to consider is what time scales do you want to graph over, too long and you don’t see what is happening now, but too short and you may only worry about an intermittent issue?

Networking Event

funkysi1701@gmail.com (funkysi1701) — Sun, 22 Mar 2015 20:00:45 +0000

I recently went to an event called ITBoss organised by Core Com Consulting . It was really good, helped partly by a free bar and great food.

The event was hosted in a pub in central Leeds, so I rushed to the train station from work to get there. And yes I was the first to arrive (Why do I always do that?). The event was targeted at IT Leaders and Managers and was limited to about 20 people.

To begin with I just chatted to the hosts and other people as we waited for everyone to arrive. Once we got going we had a short presentation about the need for Strategy. Interestingly the next day I suggested a strategy meeting with my director, so something must have sunk in from this.

The main gist of the talk was that IT leaders need to pop there head up from day to day work to look at where they are going, and this should be matched with the overall direction of the company. I agree with this, recently I have been thinking of what direction our infrastructure needs to go in, it is currently just in my head, but at least that is a start. However our company directors are poor at sharing their long term vision, so it is hard for me to know where the company is going. I need to step up and challenge this behavior, so I can deliver the changes that are needed. Not easy but that is what the best IT leaders are doing.

After this we broke for food, which was very nice. I had a chat with a couple of people. I found it interesting that one gentleman was surprised that I wanted to get into development. I think I understand why, most of the people in the room where senior managers, while I share their job titles, I don’t share their salaries. He saw development as a demotion, I see it as a promotion.

After food we had some discussion questions. Mine was the first, How to convince others of the need for change? Some interesting ideas where presented, but it is the need to work closely with others, find out their requirements which is the difficult part not anything technical. The other topics of discussion where relating to recruiting people with skills that you have no knowledge of and around the topic of security and privacy in this modern age. Both were very interesting with lots of viewpoints expressed.

I enjoyed the evening and hope to go to another at some point. Previous events I have been to have been technical, but lacked the networking that this one had. While I am not very good at networking, at this event it was easy to have a conversation or two with people.

Source Control Fail

funkysi1701@gmail.com (funkysi1701) — Mon, 16 Mar 2015 00:00:00 +0000

Every developer uses source control, it is a great tool for keeping track of changes to your code, seeing who has done what.

However I keep messing up, my use of it. I am fairly disciplined when creating new features, all my changes will get committed to source control and when I am happy I will deploy these changes to the live system. But as soon as there is a bug, especially one where the client is chasing for a fix, I will deploy a fix as soon as humanly possible on to the live system.

At first glance there is nothing wrong with what I have described but what happens the next time I deploy a new feature. Yes the bug the client was complaining about gets deployed with the new feature as the fix was never committed into source control. The client gets angry as the bug he was screaming about is back again.

Every change you make to the system MUST be committed to source control. If it isn’t that change will look like it never existed. I have worked with source control for over 5 years why do I keep making this rookie mistake over and over.

Troy Hunt has a blog post about the 10 commandments of using source control. His number two commandment is “If it’s not in source control, it doesn’t exist” He talks more about code you have written being not saved into source control, but the principal is the same for my example as his.

There are ways to automatically deploy from source control, however most of the time you don’t want your live database being rebuilt because you fixed a typo. Additional steps will need to be implemented and there is still the chance that you might want to bypass these steps to fix the urgent problem. The only way past this problem is for you and everyone on your team to be disciplined and only ever commit to source control first, and only after that deploy live (either automatically or manually)

Azure Traffic Manager

funkysi1701@gmail.com (funkysi1701) — Thu, 12 Mar 2015 20:00:45 +0000

I have spent most of the day tweaking my Azure websites. Lots of fun!

Last week unfortunately Azure had some problems and many websites that were running in the North Europe data centre were unavailable for several hours. And you guessed it my websites were hosted here.

All hosting providers are going to have downtime from time to time and this is just something you have to take on the chin. The important thing to do in times like these is communicate with your customers about what is going on and that you are doing everything you can to restore service.

However Azure has some amazing features that you can configure to help manage when downtime occurs.

Azure is Microsoft’s global cloud platform. And it really is global, there are data centres in North Europe, West Europe, Brazil, Japan, two more in Asia and five in the US. In the event of problems it is highly unlikely that more than one of these would go down at once. If all of these are unavailable, I expect the planet earth is facing some kind of cataclysmic event and the fact that my website is down is not a priority.

To take advantage of these multiple data centres, Azure has something called a Traffic Manager.

Traffic Manager has various settings but I am using it in failover mode. This means that if one website goes down, the next one is used.

All you need to do is create a traffic manager, add two or more websites to it (called endpoints) and choose a page that needs to be monitored so Azure knows which websites are up and which are down.

If you are using SSL or custom domain names, there are a few extra steps you need to do. Your custom domain name needs pointing at the traffic manager, not the individual websites. The websites themselves have three domain names, the traffic manager address, the azure address and the custom domain name. The SSL certificate can then be assigned to each website that you have added to the traffic manager.

That was easy wasn’t it, and now if a website goes down traffic manager will use the next one. While testing this, the transition to the next website was almost immediate. I did notice that if you had a browser showing the website open during a problem you sometimes got an error page, I think this was probably due to browser caching, reopening a tab or browser fixed this issue.

Tidying my desktop

funkysi1701@gmail.com (funkysi1701) — Wed, 11 Mar 2015 20:00:45 +0000

My desktop is always a mess. I constantly download files there and forget all about them.

Every now and then I copy files into sub directories, so my desktop looks sane for a day or two before it gets out of control again.

Why don’t I write a script that I can schedule to do this for me. Then my desktop will always be tidy.

I have written a few simple batch scripts, but of course the best scripting language out there at the moment is PowerShell. Lets use that.

Windows provides a nice little utility for writing scripts called the Windows PowerShell ISE, so let’s start by loading that up.

PS has lots of help included to help you, just run Get-Help [name of ps command]

To move files you can use Move-Item which works very similar to copy, specify source and destination. In my case I moved files based on their file extension.

Move-Item *.pdf folder

Now all I need to do is schedule this script to run either every day or so, or maybe every time I login or switch my computer on.

PowerShell can do lots more interesting things which hopefully I will blog about soon.

Database Deployment

funkysi1701@gmail.com (funkysi1701) — Thu, 05 Mar 2015 20:00:45 +0000

One of my jobs is adding extra features to our internal databases. There is a fair amount of risk in doing this. I could make a change to one of the queries in the database that stops the rest of the company using it as they need to. This could be a costly mistake, almost as bad as the database server being inaccessible.

One way to reduce this risk is to test, test and test the database. First I take a backup of the database and make my changes to that. I can do anything I like to my backup database without worrying that it will affect anyone else. Once I have finished making my changes and I have tested that everything is working exactly as I want, I need to apply my changes to the production database.

Until recently I made a note of every change that I have made and then made the same change to the production database. But what if I forget to apply one of my changes to production, I have a broken database on my hands, disaster!

But there is a better way to track database changes. SSDT or SQL Server Data Tools.

Create a Database project with Visual Studio, you can then point this at your live database. This will scan the database and save scripts for all the Tables, Views, Stored Procedures, Triggers and any other object in the database. This doesn’t import the data, just the structures that contain the data. So no data protection issues or problems because your database is terabytes in size.

Now comes the really cool part, Visual Studio allows you to build a copy of the database from these scripts. All you need to do is edit the scripts and test like you did before. Once you are happy with your changes and everything is working as you want, you can point your visual studio project back at your production database and it will magically deploy all your changes, or if you are as paranoid as me about breaking things, it will create a script that you can thoroughly analyse before running on your production database.

I do not know why I never thought of setting this up until recently, it will reduce the number of times I roll out my changes but forget to setup permissions, (these can be scripted) or forget that tiny change I did weeks ago but never rolled out.

And of course all the changes that you make can be saved in source control, so easy to see what changes have been made and by whom.

This process even works with complicated database structures, in my case I needed to setup three database projects that depend on each other. Dependencies can be setup, and the test databases can even be built with different names for testing on the production server, (although I wouldn’t recommend it!)