2018 on Funky Si's Blog (Dev)

Looking back at 2018

funkysi1701@gmail.com (funkysi1701) — Thu, 27 Dec 2018 00:00:00 +0000

As 2018 starts to draw to a close let’s look at some of the highlights from the past year.

New Home - I started 2018 with the goal of buying my own home and I managed it. At the end of October I actually moved in but it was months of looking at houses, speaking to estate agents, saving, packing and unpacking. There is more I want to do to my home in 2019 so this is only really the beginning.
New Job - Another goal was to kick my career up a notch and this happened in June. It has been great working as part of a large team and I have learnt loads, there is much more I want to learn and contribute to so 2019 should be great on that front.
New Car - Not a goal that I achieved as unfortunately in October I was involved in a car accident, no one was seriously hurt but this was a horrible experience to go through. My car was written off which gave me an excuse to buy a new car.
See more of family was a goal, I am not sure if I really achieved this. However family have been around loads for us during our move and we do live closer so hopefully I can work on this some more.
Celebrate 5 years of marriage. It hasn’t been an easy year for me and the wife due to moving house and all the stress that involved plus with two children she was diagnosed with Postnatal Depression which as a family we are still dealing with. All that said we had a fab weekend away and we so need to do something similar again soon.
Family. 2018 has included many great times with my two boys. It’s been amazing seeing them grow, in September we had a thanksgiving service for them and in July we had our annual holiday.
Lightning Talk. This is the only thing I mentioned as a goal that didn’t happen, it’s on my 2019 list so hopefully I can kick myself into action but with everything else I achieved something had to slip.

Run SQL Server in a Linux container

funkysi1701@gmail.com (funkysi1701) — Mon, 05 Nov 2018 00:00:00 +0000

Recently I have been investigating what all the fuss is about Docker and it has been well worth my time as Docker is pretty awesome for automating stuff.

My development environment has typically required installing SQL Server. SQL is a bit of a beast with lots of options and takes time to setup how you want.

However since Microsoft have now created a version of SQL Server that runs on Linux you can run SQL Server in a Linux container with only a few commands.

I am going to assume you already have Docker for windows installed on your development machine. If not head over to Docker and find out how.

The Microsoft guide to setting up SQL Server in a Linux container can be found here .

First you need to download the image. In a powershell window run:

docker pull mcr.microsoft.com/mssql/server:2017-latest

This downloads the latest sql server image.

To run this image run the following:

docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=password"
-p 1433:1433 --name sql
-d mcr.microsoft.com/mssql/server:2017-latest

To run a SQL Server image you are required to accept the terms and conditions and set a default sa password. These are added as environment variables with the -e flag.

You also need to set the ports that your container will run on (1433 is the default SQL port) and give your container a name, in this case “sql”.

If you have already installed SQL Server you will not be able to run the container on the same port as your local install. To solve this you can select a different port.

docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=password"
-p 1434:1433 --name sql
-d mcr.microsoft.com/mssql/server:2017-latest

-p 1434:1433 maps the 1433 port on the container to port 1434 of your local environment.

Once you have run this command you can connect SQL Server Management Studio (SSMS) to (local) or (local),1434 if you are using a different port using the credentials you provided and execute any SQL you like.

If your development environment requires windows authentication this of course is not for you, if it doesn’t you are good to go.

The development environment I have been using has various powershell scripts for setting things up. These assume windows auth. However I have adapted them to take custom credentials.

$credential = Get-Credential $server.ConnectionContext.LoginSecure=$false
$server.ConnectionContext.set_Login($credential.UserName)
$server.ConnectionContext.set_SecurePassword($credential.Password)

The Get-Credential command creates a dialog where you can enter SQL credentials, this is then stored in a variable and used in the rest of the script.

How do I restore a backup file to my container?

Run:

docker exec -it sql mkdir /var/opt/mssql/backup
docker cp database.bak sql:/var/opt/mssql/backup

This creates a backup folder and copies a backup file from your local environment to the container. You can then use management studio to restore the backup file (or you could write a sql script to do it). One thing to note when restoring databases, make sure the files are restored to Linux locations not windows locations.

The only issues I have encountered so far are the lack of support for SSIS packages and no windows auth. There are sql server windows images available which I haven’t tried yet which may work better with some of these options.

2018 The Story So far

funkysi1701@gmail.com (funkysi1701) — Sat, 28 Jul 2018 00:00:00 +0000

Remember me? I used to write blog posts but somehow life got in the way. So much has happened this year, lets have a look at my goals for 2018 and see how far we have got with them.

Buy a house

My first goal for 2018 was to buy a house. Well so much progress has been made with this goal.

We looked at a few houses, and have made an offer, had it accepted and are waiting for the last few things to get arranged before we start arranging to move in.

The location we decided upon was Thorne . This is about an hour from York where most of our friends are and my wife has lived all her life, and also an hour from where my family live. I have tried my best to balance location and size of property and we think our new home is going to be a good balance but only time will tell.

I have a bit of a mammoth time ahead of me getting everything we own packed up and moved to our new home so wish me luck, but we will get there it is just going to be hard work.

New Job

In June 2018 I started a new job. This was slightly later than I had hoped but I believe it has been worth the wait.

My new job has lots of perks. Finish early on a Friday (1pm), On site canteen, Cool air-conditioned office, Flexible working hours, Part of a big development team.

The main benefit is that I am part of a big development team. I have a huge list of things to learn both in terms of technology and how large development teams function. The team is about to finish the first sprint that I have contributed to. My contributions have been very small. A bug fix here, a tweak of an API there. But the important thing is that I am making a contribution and learning along the way.

I have also started asking questions about deployment and how this could be automated. Something that interests me, especially with the sysadmin and devops background. I have been tasked with learning docker , so expect some blog posts on this topic in the near future.

See more of family

This one I am going to count as a miss. I have seen family a few times this year, but my efforts have been more on the first two items in this list. Hopefully this will change as things settle down.

Celebrate 5 years of Marriage

Me and the wife had a great weekend away to celebrate our wedding anniversary.

Celebrate my two children

Not happened yet but it is booked and we just need to arrange a few more details.

Lightning Talk

I have made no effort to do a talk and this is going to be a goal for 2019 I think. Most of the reason is concentrating on the other areas above, but also because I am not a natural speaker and I don’t like stepping out of my comfort zone. But I will do it, I just need to wait for other things to calm down a bit first.

Family Holiday

We had a great week away at the start of July. The weather was really great (and still is) and we had lots of time doing things as a family.

Casting and Converting between types

funkysi1701@gmail.com (funkysi1701) — Mon, 07 May 2018 00:00:00 +0000

Recently I was asked how to convert a number to a string. Let’s look at a few ways of approaching this problem.

Most objects in c# have a method called ToString() which displays the string representation of that object. This is because of inheritance, all objects inherit from System.Object which defines ToString().

Int32 is a struct so it inherits from System.ValueType which also inherits from System.Object

so in code

int a = 9;
string b = a.ToString();

Now let’s look at the reverse. However the reverse runs the risk of throwing an error, let’s look at why.

string b = "9";
string c = "a";
string d = "two";

All are valid strings but only one can be converted to a number. Use the TryParse method to convert to a number.

int.TryParse("9", out int e);

TryParse will not throw an exception if the conversion fails, if it succeeds variable e will contain the result. Note an earlier version of c# required you to define the out parameter before using it with TryParse.

int.Parse exists to do the same thing however it will throw exceptions if a conversion is not possible. The same is true if you use Convert.ToInt32(“two”);

Casting

Casting is a way to explicitly telling the compiler that a type is actually another type and you are aware data loss will occur.

double x = 4.5;
int y = (int)x;

However it is not possible to cast a string to a number format as a string can contain any character not just number characters.

Let’s Encrypt is awesome

funkysi1701@gmail.com (funkysi1701) — Mon, 30 Apr 2018 00:00:00 +0000

Let’s Encrypt is a free way to get a SSL certificate onto your website and until recently I had never tried it. It is very easy and I think it is awesome.

IIS is the web server software the Microsoft include with Windows 10 and Windows Server. I have it installed on my laptop and it displays the default IIS page.

It is not really a good idea to host websites on your laptop, use a dedicated web server, or host with a hosting company, however the techniques are the same and it gives me something to write about!

In order to point a domain name at what IIS on my machine was serving up I did the following:

Do a google search for “whats my IP”, this will return your public IP. Most residential ISPs use dynamic IPs so it may change over time, (which is another reason not to host a website on your laptop!)
Add an A record on a domain with the IP address you have just got
Your public IP most likely points at your router not your laptop so enable port forwarding of port 80 and port 443 to the internal IP of your laptop (something like 192.168.0.11 etc)

Now comes the fun Let’s Encrypt stuff!

First you need a Let’s Encrypt client, there are a lot of them out there mostly for linux flavours, however a bit of googling found a windows one. Go to https://github.com/PKISharp/win-acme/releases and download the zip file and unzip it.

Run the executable from the zip file and follow the onscreen prompts.

Press N to create a new certificate.

Then press 1 to bind to single website found in your IIS setup

And now magically Let’s Encrypt knows what you have setup in IIS.

Now all you need to do is enter an email address incase a renewal fails and agree to the let’s encrypt terms and you are all setup.

How awesome and easy is that for getting your websites working with a SSL certificate. If you have IIS configured on a server, give it a try and you can SSL all your things.

DNS for Developers

funkysi1701@gmail.com (funkysi1701) — Mon, 09 Apr 2018 00:00:00 +0000

DNS is the backbone of the internet and as such I believe every developer should know something about the basics and not just leave it for the sysadmin to sort.

What is DNS?

DNS or Domain Name System is what translates Domain names to IP addresses and vice versa.

Wait what is an IP address and what are domain names?

You do realise this is a developer blog? An IP address is a unique address on the internet and a domain name is a user friendly label for one or more of these.

An example might be google.com which for me resolves to 216.58.204.14

How it works

When your browser makes a request to google.com it makes a request to your ISPs DNS Servers. This resolves google.com to 216.58.204.14

In more detail your ISPs DNS server will forward the DNS query to another DNS server and will cache the results for a set amount of time. This is the TTL or Time To Live. Next time the ISP DNS Server will be able to reply directly without needing to forward requests.

This forwarding and caching is what makes making a DNS change not instantaneous. The TTL needs to be reached so that no results are still being fetched from the cache of DNS servers across the globe.

DNS Records

Now we know roughly how DNS works let’s look at the most common type of records

A

A (Host) records are the most simple records which translate domain names to IPs

eg www.google.com to 216.58.204.14

CNAME

A CNAME (Canonical Name) record is different to an A record in that it maps a domain name to another domain name when no A record exists.

eg www.google.com to somethingelse.google.com

Typically Azure makes use of CNAMEs for many of its services especially adding a custom domain name

MX

MX stands for Mail Exchange and is used for configuring email

Name Server

Every domain has a number of Name Servers which tells you what servers control the DNS settings for that domain. If you change your Name Servers then the new Name servers will be where you can change your DNS settings.

If you want to use a service like DNSimple instead of 123reg or where ever you registered your domain then all you need to do is change your Name servers.

AAAA

Like A record but for ipv6

What next?

Want to put some different DNS records into practise? Buy a domain name and publish some content to it. Check out my previous post about programmatically adding records . Want an SSL certificate? Get a wildcard one and then you can apply it to any subdomain you add to your domain.

If you have a new website you want to publish consider which of the following is better:

https://www.example.com/newsite

https://newsite.example.com

I much prefer the second option, it looks cleaner, there is no potential conflict with the parent site, no subfolder issues between production and development.

Code Reviews

funkysi1701@gmail.com (funkysi1701) — Mon, 02 Apr 2018 00:00:00 +0000

Reviewing code is a great habit to get into. Code reviews help share knowledge between your team members and help catch bugs before they get into production. But how do you get into the habit of reviewing and avoid the we don’t have time to do this mentality?

Visual Studio Team Services (VSTS) has some great options that can help make code reviews second nature.

Pull Requests

A lot of source control systems have the concept of pull requests. This is where you request others to review your code usually in a branch and if they approve it, merge it into a main branch.

To create a pull request in VSTS go to the Code section and select Pull Requests. Often VSTS will make a suggestion of what branch to make a pull request for, if you don’t see this just click the New Pull request button.

Select a branch you want to merge from and a branch that should be merged into (usually you merge into master from a feature branch). Give your pull request a title and description and select who should review your code, this can either be an individual or a group of people. You can also review all the changes that will be reviewed so you can make any last minute changes before it is reviewed.

Now if you are anything like me you want your code merged in as soon as you have created your pull request and there is nothing stopping you reviewing your own code and clicking approve and merge on your own pull request. However branch policies is a way around this problem.

Branch Policies

Branch policies allow you to specify how your code gets merged in.

Go to the list of branches in VSTS and select branch policy and you will see a whole host of options to customise the merge process. If you do this on the master branch you will not be able to commit any changes to master without it going through a pull request.

The first option enables you to select how many reviewers are needed on your code. If no one else works on your project best not setting this, but for everyone else setting at least one person to review your code is a great practice.

Next you can ensure that your pull request is linked to a work item, this helps keep ensure you are actually fixing issues and not just making change for the sake of it.

Check for comment resolution is a good setting to enable. This ensures that if your reviewer has commented about you needing to change this line here, it ensures that you do.

Enforce merge strategy allows you to choose between fast forward merge or squash merge.

Build validation enables the code to be built using a build definition you have configured. This is a great way to check code builds or tests pass before it gets merged in.

The last two options allow you to specify code reviewers and third party external services.

Android Development Options

funkysi1701@gmail.com (funkysi1701) — Mon, 26 Mar 2018 00:00:00 +0000

A friend asked me how to get started in Android Development and I thought I might have a go at answering that question here.

I am by no means an expert in Android development, I do have an app in the play store so I know something.

Manifest File

This is probably the easiest option and also doesn’t actually create an android app so I am not sure if it should be included in this list of not.

If you have a website and you want to create an app for that you could just create a manifest file and add this to your website.

Once your website has a manifest file, if you visit your website using a mobile phone or tablet you will get the option to add a shortcut to the home screen. You then have an app like experience in that you can click an icon to launch your website.

A manifest file is a simple text file which specifies a few settings like the icon size, filename, what page loads when clicked and name of your “app”

More information

Visual Studio

This is the option I know most about as is what I have used.

If you are familiar with Visual Studio you can use the Xamarin Forms software to create your app in C#. Xamarin Forms allows you to easily create cross platform apps that run on Android, IOS and windows phone. So far I have only experimented with Android but it should be relatively easy to extend my code to run on other platforms.

Xamarin Forms allows you to write one a single codebase that can be compiled to run on the different platforms. Xamarin requires the use of XAML a XML like markup language for designing UI elements.

More Information on Visual Studio , Xamarin Forms

Android Studio

I don’t know much about this option so do correct me if I don’t get the details correct.

Android Studio can be downloaded from Google this allows you to create java code to run directly on an android device. From what I know this is fairly similar experience to Visual Studio but instead of writing your code in C# you use Android Studio and write it directly in java.

More information

Cordova

Cordova allows you to use HTML, CSS and javascript to create cross platform apps. I have no idea why I haven’t heard of this technology until today as it sounds very flexible especially if you know a little bit of javascript.

More information

To summarize there are lots of different options available to create an android app. What you choose depends on what you want to build, what language and experience you have and if your app needs to be cross platform.

Tips for Developing Yourself

funkysi1701@gmail.com (funkysi1701) — Mon, 19 Mar 2018 00:00:00 +0000

For a while I have been mentoring a friend and I thought I might share some top tips I have implemented in my career so far.

Hard Work

At school I was told: “Only in the dictionary does Success come before Work“. Yes it is a bit of a corny saying however it has stuck in my head and it’s true. If you want to get anywhere you need to work at it. For many years I just drifted along, but as soon as I knuckled down and worked hard my career started going somewhere.

Write a Blog

Writing is an important skill that you need to work at to improve. If you commit to writing one blog post a week this will help to improve this skill. Even after 150+ blog posts I still feel I need to improve. Not only that it will also help others learn something from you and shows a willingness to contribute to the community. If a Hiring Manager reads your blog he can see some of the stuff you know or care about and may help give you the edge over more experienced candidates.

Soft Skills

This is one I need to work on but you should concentrate on the so called “Soft Skills“. Soft Skills are the skills you have for dealing with others. Are you good at talking to people? Or good at getting requirements out of business owners? If so you have some soft skills, don’t underestimate the value that these bring to a team and keep developing them.

Build a Side Project

Start building a side project, it doesn’t matter what it is but start building something. I started building a Xamarin app but I have learnt loads of other connected things while doing this, ViewModels, APIs, Build and Deployment processes, Azure Functions. It also gives you ammunition for things to blog about. If you are short of ideas try solving a problem or rebuild something you use.

Use your spare time

During the day there are moments you can reclaim for learning stuff. There are lots of podcast which discuss useful development topics, listen to these while driving to work. Subscribe to pluralsight and listen to this while washing up. Don’t get too concerned with learning everything, however think how much more you are learning than not listening at all.

I’m not ready!

We are never ready to take that next step, it is so easy to make excuses like I need to learn x, or know y. The only way to know if you are ready is to try and to keep trying. Iterate as you go so you keep improving yourself. If you wait before you start trying you will end up waiting forever. If you start trying now you will make some small progress and have a better idea of what you need to do to achieve your goals.

Heroes

funkysi1701@gmail.com (funkysi1701) — Fri, 16 Mar 2018 00:00:00 +0000

Who was your hero growing up? Mine was Stephen Hawking.

What is a hero? A lot is said in the media about “Heroes”. Often the sportsperson of the hour is described as a hero. I don’t like this definition and I think there is far more to being a hero than that. One definition I saw describes a hero as A person who is admired for their courage, outstanding achievements, or noble qualities.

Why Hawking? As a kid I thought I was pretty smart (Not really just above average), I liked Star Trek and science so Hawking was a pretty obvious choice. I remember reading all about Hawking and his life fascinated me. A brilliant man was struck down by motor neurone disease in his early twenties, however this did not stop him and he completed his doctorate and became the famous physicist we all know and love.

My physics career was fairly short lived ending with an undergraduate course at University of York, however I really enjoyed learning the concepts that he described about space and time and was definitely a driving factor for me to learn as much as I did about Physics.

How heroic is it to be given 2 years to live and then not only survive but to expand our understanding of the universe without being able to speak or move without the aid of technology?

I can not imagine what it must be like to not be able to communicate without the use of a computer, but this has been Stephen Hawkings world for over 30 years. His computer system has undergone various upgrades over the years however his synthetic computer voice has continued as he identified with it and prefered it.

Stephen Hawking had an immense intellect, I would probably describe him as one of the cleverest people on the planet.

Who are your heroes? What kind of person is your hero, is it a scientist or engineer? The world contains some very clever people who will inspire and excite young people to strive towards something. I am hopeful that my sons can be inspired even if it’s only a small way by someone like Stephen Hawking.

New version of Pwned Pass

funkysi1701@gmail.com (funkysi1701) — Mon, 05 Mar 2018 00:00:00 +0000

A new version of Pwned Pass is available from Google Play.

A couple of weeks ago Troy Hunt released V2 of Pwned Pass onto his haveibeenpwned website. There are now over half a billion passwords that have appeared in data breaches for you to search.

This time Troy has included information about how many times a password has appeared in his data. So “password” returns a value of 3,303,003 so is a really, really bad choice of password to use. However the password “windows 10” has only appeared twice so is much better, however I would still recommend avoiding using it.

One of the great things about the internet is that on the day of release I was tweeting Troy so I could get my android app updated to take advantage of these new features. I even helped him identify an issue with the documentation on his site. He has also kindly add a link to my app onto his website.

That conversation and link has lead to a massive increase in app downloads. Before I was going steady on about 14 installs, now I have 86 active installs. I am not sure if this will increase anymore but if you are reading this because you downloaded my app, then thank you.

My app has four main features:

Password Check – this allows you to search a password and see if it has appeared in a data breach and also the number of times that password has appeared.

Email Check – this allows you to search your email address and see which data breach it has been involved in.

List of Data Breaches – this lists the data breaches from haveibeenpwned.com you can also sort by Date Added, Number of accounts and name.

Calendar of Breaches – this shows a github like chart of what days breaches are added

If you like my app do let me know. I have received a few ratings on google play it would be great to get a few more. If you want me to add a feature or have ideas of how I could improve it let me know as well.

To download take a look at https://play.google.com/store/apps/details?id=pwnedpasswords.pwnedpasswords

Refactoring if statements

funkysi1701@gmail.com (funkysi1701) — Mon, 26 Feb 2018 00:00:00 +0000

The code base I am working on contains a huge if block. By huge I mean 77 if statements one after the other, each if checks to see what page id you are on and loads different content. This is not easy to maintain and I want to refactor it.

One option would be to replace the if statements with a switch block. However this is just as unmanageable as the huge if block. Lets look at a better option.

Polymorphism is where you create a base class and then create sub classes from it. In my case I created an interface IPage with a single method CreateContent.

public interface IPage
{
 string CreatePageContent();
}

and then create 77 classes for each page which implemented this single method.

Now comes the fun bit how do I call the correct page class from my original code?

I created a dictionary than maps page ids to the class names.

public static Dictionary<PageIds, Type> PageIdToClass = new Dictionary<PageIds, Type>()
{
 {
 PageIds.HomePage,
 typeof(HomePage)
 },
 {
 PageIds.ContactPage,
 typeof(ContactPage)
 },
 //etc 
}

This is the one step I am not 100% happy with as I think it may be possible to remove or simplify this step.

Now I have a way to map ids to classes I can write a class to do this.

public class MyPage
{
 IPage _repo;

 public MyPage(int pageId)
 {
 PageIds p = (PageIds)pageId;
 Type t = PageIdToClass[p];
 ConstructorInfo constructor = t.GetConstructor(new Type[] { });
 _repo = (IPage)constructor.Invoke(null);
 }

 public string Create()
 {
 return _repo.CreatePageContent();
 }
}

So in my constructor I take the pageId and pass it to my dictionary to get which subclass to load. I then get its Constructor and invoke it.

Now I can remove the huge if block and replace it with a single line of code.

var page = new MyPage(pageId);

On the face of it this change might look like a lot of work for not much gain as we started off with one file and now we have the original file, an interface, 77 subclasses and the MyPage class. However the original file is a lot more manageable and each sub class can be altered independently of each other.

This is a big step towards making this code more maintainable, there is always more that can be done but that can wait for another day.

Chrome distrusts SSL Certificates

funkysi1701@gmail.com (funkysi1701) — Mon, 19 Feb 2018 00:00:00 +0000

One of the websites I have been working on has been displaying an error in the console. The error reads as follows.

The SSL certificate used to load resources from https://example.com will be distrusted in M70. Once distrusted, users will be prevented from loading these resources. See https://g.co/chrome/symantecpkicerts for more information.

But what does this mean? Well let’s start by looking at the link provided.

In January 2017 it was revealed that Certificate Authorities run by Symantec which include Thawte, VeriSign, Equifax, GeoTrust, and RapidSSL had been issuing certificates that did not comply with baseline standards.

Starting with Chrome 66, Google has decided to remove trust for these certificates. Chrome 66 is due for release around 17th April. My error mentions M70 so what does that refer to?

Chrome 70 which is due to be released in October 2018 will removed the trust for another batch of Symantec certificates.

If you are getting one of these errors because you are using a certificate that is going to be distrusted what will your site look like in Chrome 66 or Chrome 70?

Well Chrome 66 is now in the dev channel so we can give it a try.

Not very nice for your users is it? Now is the time to order a new SSL certificate to avoid this happening to your site.

I first saw this error a few months ago and have been reading up about it and waiting for Chrome 66 to reach the dev channel so I could test what it did to my site. However now that I have Chrome 66 installed I spotted the intranet for the company I work for is also affected. I do not directly work on the intranet so I notified the security team that they may want to look into this.

Unfortunately the response I received has been that Google needs to fix this before Chrome 66 is released. I am not criticising my employer or the security team, however this isn’t something Google can just “ fix “.

The certificates issued were issued by a CA that had issues so in order to maintain the trustworthiness of all certificates Google had little choice but to distrust them. Google and security experts need to be making more of a fuss about this and I am joining in on making a fuss by writing this blog. Scott Helme estimates that there are about 7000 websites which may be affected by the M66 and M70 distrusts.

Content Security Policies

funkysi1701@gmail.com (funkysi1701) — Mon, 12 Feb 2018 00:00:00 +0000

A content Security Policy or CSP is a HTTP response header that defines what sources of content can be loaded on a web page. It is a way to combat Cross Site Scripting (XSS) attacks.

What is a XSS attack then?

When you load a webpage it also loads various other resources like images, some css style sheets, various javascript files that you want to run and probably many other things.

How do you know that you can trust all of these things? If you created them and they live under you control then the answer is probably yes. However these days you will probably want to use resources from across the internet, like youtube videos, google analytics, disqus comments, jquery libraries from a cdn etc and you can’t be sure exactly what they are doing.

Imagine you had a page which you could add any text into a form which would then be displayed. A malicious user could add evil javascript or get the browser to load evil code from anywhere on the internet.

CSP to the rescue!

A CSP allows the browser to only load from sources that you specify. You could specify that resources from your own site will load but the evil script will not.

Let’s look at some examples

Content-Security-Policy: script-src 'self'

This allows <script> tags to only load from the current webhost. script-src is not the only keyword you can use, let’s look at some of the others.

script-src – control what <script> tags will load
style-src – control what css will load
img-src – control what images will load
frame-src – control what frames will load
font-src – control what fonts will load
object-src – control what object tags will load
connect-src – control what resources a script can connect to
media-src – controls what media (audio/video) will load
default-src – if no specific rule exists then the default directive will run

Content-Security-Policy: default-src https

This allows any content to be loaded from any site as long as it comes from a secure (https) site

Content-Security-Policy: default-src https://example.com

This allows any content to be loaded from https://example.com only.

How do I use this on my site?

I have added CSPs into my web.config which works great for my .Net Framework code.

 <system.webServer> <httpProtocol> <customHeaders> <add name="Content-Security-Policy" value="default-src https://example.com" /> </customHeaders> </httpProtocol> </system.webServer>

For .net core it is a bit more complex as you don’t tend to use web.config files, however check out Anthony Chu’s post , which has a solution to that problem.

Report Only

One last thing about CSPs to mention is the Report Only flag.

Content-Security-Policy-Report-Only

This does the same as the above but doesn’t enforce anything, so you can fix any problems before you break anything.

To view your issues just look in the developer tools in your favourite browser. Or you can configure all your reports to be collated in one place with a report-uri directive.

Content-Security-Policy: default-src https://example.com; report-uri https://example.report-uri.com/r/d/csp/reportOnly;

Scott Helme and Troy Hunt have a site called report-uri which offer a service for collating and viewing all your CSP violations so check it out if you want to know more about CSPs.

Pluralsight

funkysi1701@gmail.com (funkysi1701) — Mon, 05 Feb 2018 00:00:00 +0000

Earlier this year I signed up for pluralsight. If you want to sign up to pluralsight as well use this link https://referral.pluralsight.com/mQd8BJ4 to get money off.

Pluralsight is a website that sells training videos on a wide variety of technical topics. You sign up for a monthly or annual subscription and you can watch over 6000 courses whenever and where ever you like.

I am not a big fan of videos as if I am sat in front of a screen I would rather be building something, however I do spend a lot of time listening to podcasts. So I have decided to convert this time to listening to pluralsight videos.

If we assume that by not seeing the visual portion of the videos you lose out on 75% of the learning (This is just a guesstimate I am not sure what the exact figure is or how you would calculate it). However this is still 25% more learning than if I hadn’t signed up at all.

Since I signed up I have listened to over 27 hours of technical videos. These have covered topics like C#, Security, Xamarin, MVVM, Interfaces, Getting Involved and Clean architecture. The number one thing I have learnt is that there is a lot that I still don’t understand.

That said I would like to think that since starting to listen to pluralsight concepts have gone from being unknown unknowns (I don’t know about them and I don’t understand them) to known unknowns (I know about them but don’t yet fully understand them)

To keep track of your learning pluralsight has some tests which you can complete. I am particularly proud of my score on the Azure test which ranks me as an expert.

Why not check out what you can learn from pluralsight?

Flexible architecture and interfaces

funkysi1701@gmail.com (funkysi1701) — Mon, 29 Jan 2018 00:00:00 +0000

I have blogged a few times about interfaces , and how useful they are for producing good quality maintainable code. Let’s look at a problem and the solution I came up with which I am quite proud of.

As previously mentioned I am in the process of moving images from AWS to Azure blob storage. Now that the actual files themselves have been moved I need to change the code that references them.

Now I could find all the code that uses the AWS API and replace it with the Azure API but I am not very good at predicting the future, we may stay on Azure for a while, we may move to AWS or Google Cloud, or we may want to go back to files sitting on a server.

Lets try and code a solution that is as flexible as possible. As you have probably guessed I am going to create an interface.

At first I thought about creating an interface called ICloudStorage , however this isn’t flexible enough as what happens if we go back to sticking files on a server so instead I created IStorage.

I created three classes that implemented IStorage, AWSStorage , AzureStorage and mostly for testing at the moment FileStorage. I then created a class Storage that would call these three classes. Initially I created it like this

public class Storage
{
 private IStorage _repo;
 public Storage(IStorage repo)
 {
 _repo = repo;
 }
}

However this would require I call it like Storage(new AzureStorage()) and I would need to know everywhere in my code which implementation I want to use. This isn’t too bad as when we change from AWS to Azure we would need to do a find and replace throughout the code and replace all AWSStorage and make them AzureStorage.

However we can do better than that.

public class Storage
{
 private IStorage _repo;
 public Storage()
 {
 Type obj = Type.GetType(ConfigurationManager.AppSettings["DefaultStorageRepository"]);
 ConstructorInfo constructor = obj.GetConstructor(new Type[] { });
 _repo = (IStorage)constructor.Invoke(null);
 }
}

This code will read from the web.config which implementation to use and that will decide which class to call. This means that to change from AWS to Azure we do not need to redeploy any code, all we need to do is change the web.config.

Let’s look at the three lines and see if we can understand what is happening.

Type.GetType() looks straight forward and gets the type from the web.config

obj.GetConstructor() This gets the constructor for the type we have just found.

constructor.Invoke This then invokes the constructor and it then gets cast to the interface so can be used by the _repo variable.

This is all fairly simple and makes sense, however it has produced some very flexible code and allows the code to be extended without recompiling.

Let’s look at a hypothetical example. We want to add support for Google Cloud Storage. All we need to do is create a class library which implements the IStorage interface, place the compiled binary in the website and update the web.config to reference it. I haven’t tried this hypothetical example so it might be more complex than I think but in theory it should work.

I am pretty excited at how flexible this code can be, hopefully I will use code like this more often now I understand it.

Moving files into blob storage

funkysi1701@gmail.com (funkysi1701) — Mon, 22 Jan 2018 00:00:00 +0000

We are in the process of moving our companies websites onto the Azure platform. One of the challenges was to move image files out of the website project into blob storage. This week I have moved 150,000 of them.

One thing I keep banging on about is that your source code should not contain data. If it does every time you do a deployment you need to consider where these images are located and ensure you don’t overwrite or loose any. It also goes without saying that deployments of a few Mb are a lot quicker than deployments of 100s of Mb.

Azure blob storage also gives you advantages like distributing storage across multiple datacenters which would be impossible with traditional files on a server.

So now that we have established that this is a good idea lets look at how we could move large amounts of data. In my case all the filenames are stored in a SQL database so the plan of action was to simply loop through the files in the database, download from current storage (either locally or other cloud storage), upload to Azure and tidy up afterwards. Due to the number of images I am going to update the database and mark when a file has been processed so I can do the move over several days.

This is my code

var source = "https://example.com/images/";
var tmp = Server.MapPath("~/tmp/");
if (!Directory.Exists(tmp))
{
 Directory.CreateDirectory(tmp);
}
var fixturePhotos = db.Images.Where(x => x.Moved == null || x.Moved == 0).Take(id);

foreach (var photo in fixturePhotos)
{
 try
 {
 string path = getFilePath(photo.FileName);

 if (!Directory.Exists(Server.MapPath("~/tmp/" + path)))
 {
 Directory.CreateDirectory(Server.MapPath("~/tmp/" + path));
 }
 WebClient WebClient = new WebClient();
 WebClient.DownloadFile(source + photo.FileName, tmp + photo.FileName);
 FileUploader f = new FileUploader(tmp + photo.FileName, photo.FileName);
 System.IO.File.Delete(tmp + photo.FileName);
 photo.Moved = 1;
 }
 catch
 {
 photo.Moved = 0;
 }
}
db.SaveChanges();
if (Directory.Exists(tmp))
{
 Directory.Delete(tmp, true);
}

First of all I create a tmp folder in the root of my website if it doesn’t exist to store my images temporarily.

I then use an entity framework model to query the database that haven’t been moved, and I use the take() method to limit how many results I process. (I have been passing in 1000 at a time)

I then use a foreach loop over all these files to perform the following actions.

Create additional subfolders if the filename variable stored in the database isn’t actually a filename but a filepath, note you will have to split filename and filepath which I haven’t included code for here.
Download file from the original url and save into the temporary folder
Upload to Azure
Delete temporary file
Update database giving a success or fail

Once the foreach is finished I commit the database changes and delete the temporary folder. I am sure there must be other ways to do this transfer but this was quick and easy to setup and now I have a copy of all the files in Azure storage so I can test out other issues with my website.

One last tip about how to schedule this code. I called the above code from a MVC controller and then wrote a Azure Function to call this code on a schedule.

Website UI Testing

funkysi1701@gmail.com (funkysi1701) — Mon, 15 Jan 2018 00:00:00 +0000

Last week I looked at testing the UI of mobile apps , this week lets look at how we could do a similar thing for websites.

Testing the user interface is not an excuse for a lack of unit tests . Testing the user interface takes longer so for keep creating your small unit tests that can be run after ever build. That said lets look at how you create a UI test.

Create a Unit Test project as normal. Now install the following nuget packages

Selenium.WebDriver.ChromeDriverSelenium.WebDriverSelenium.WebDriver.PhantomJS.Xplatform

I am going to be using Selenium to achieve my website testing and I am going to concentrate on the Chrome browser. However packages exist for other browsers so have a look at the following and I expect there are others as well.

Selenium.WebDriver.IEDriverSelenium.Firefox.WebDriver

Selenium started life as a plugin for Firefox to help create automated tests, however the latest version of Firefox is not compatible with the plugin as I write this. I have not had to install any plugins or extensions to my browsers to achieve my testing.

Enough talk lets write a test. First we create two instance variables to store the baseURL and the driver for the browser you are using.

private string baseURL = "https://www.example.com/";
private RemoteWebDriver driver;

Next we need to set things up for the test to run. This creates an instance of the chrome driver, maximizes the window and sets it to wait 30 seconds before timing out.

[TestInitialize()]
public void MyTestInitialize()
{
 driver = new ChromeDriver();
 driver.Manage().Window.Maximize();
 driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(30));
}

Now comes the actual test. We navigate to a URL and then compare the title of the page loaded with a know value with a Assert statement like you would find in a unit test.

[TestMethod]
public void CheckBrowserTitle()
{
 driver.Navigate().GoToUrl(this.baseURL);
 Assert.AreEqual("Home Page", driver.Title);
}

Finally we need to tidy up after ourselves.

[TestCleanup()]
public void MyTestCleanup()
{
 driver.Quit();
}

If you are used to writing tests you will know that the are usually constructed in three sections Arrange, Act and Assert. The Arrange is done in the initialize method, which makes the actual test much simpler, the first line does the Act and the last line does the Assert.

Now we have written a simple test lets look at something more complex.

driver.FindElementByLinkTest("click")

This finds any Link on the page which is click. Be careful as the string need to be exactly what appears on screen it may well be easier to specify by id or class or something that doesn’t change as often. By adding .click() on the end of this command Selenium will click on the link and you can navigate to a new page.

What about submitting form data? Well you can find the element you want to fill in and add .SendKeys(“example text”) or .Submit() and this will fill in and submit form data.

What about a screenshot?

Screenshot ss = driver.GetScreenshot();
ss.SaveAsFile("test.jpg",System.Drawing.Imaging.ImageFormat.Jpeg);

I have only just started playing around with web UI tests but you can see there is a fair bit you can do.

Mobile App UI Testing

funkysi1701@gmail.com (funkysi1701) — Mon, 08 Jan 2018 00:00:00 +0000

Since I started creating an android app I have been writing simple UI tests.

I have been taking advantage of the Visual Studio App Center which allows you to test against hundreds of different devices in the Test Cloud.

In order to write a UI test create a UI Test App, this makes use of the nuget package Xamarin UI Test. By default you will now have a test called AppLaunches which will take a screenshot of you app after it starts.

You can now run this test against any device from Visual Studio assuming you have it physically plugged into your machine. However, how do you run against the Test Cloud?

To run against the Test Cloud you need to first install node.js. Now you can install the appcenter cli with the following command.

**npm** install -g appcenter-cli

To run the tests in the Test Cloud run the following command

**appcenter** test run uitest --app "<username>/<appname>" --devices "<username>/<deviceset>" --app-path _pathToFile.apk_ --test-series "master" --locale "en_US" --build-dir _pathToUITestBuildDir_

where is your appcenter username, is the name of your app in appcenter and is the group of apps you have created in app center to test against.

Look at device sets in the test section of the appcenter and click the new device set button. You can then search for any device you like and add it to a set, as I write this there are 244 devices you can test against.

It is currently not possible within app center to run tests against a new build, however if you build your app in VSTS as well you can create build or release step that runs it. As the Visual Studio app center is still under development I wouldn’t be surprised if it is added at some point.

In VSTS look for Mobile Center Test in the definitions and you can specify the same variables as specified in the command line above.

Now how do you actually write a useful UI test? I mentioned above you get a default test which contains the following code, this takes a screenshot of your app. However you don’t have to include this when using the Test Cloud as screenshots are included for free.

app.Screenshot("First screen.");

Lets look at what else is included in app

app.Repl();

This starts an interactive REPL (Read-Eval-Print-Loop) which lets you explore what is on screen in your app and pauses execution. I don’t include this in my tests, however I do make use of it to explore what is on screen and what tests I might make use of.

app.Tap(c => c.Marked("Button"));

This taps an element on screen called Button. There is also a method called TapCoordinates which would allow you to click anywhere you like.

app.WaitForElement(c => c.Marked("View"));

After clicking a button you are probably going to want to wait for the app to load extra data or a new screen. The WaitForElement waits for an element to appear on screen. There are also methods that wait a period of time or wait until an element no longer exists.

These are the main methods I have used so far, however there is an extensive list including methods for scrolling, swiping, pinching and adjusting the volume buttons. So you should be able to test all manner of app functionality and if you make use of the Test Clouds will know which devices are causing problems.

Lets see what 2018 can do!

funkysi1701@gmail.com (funkysi1701) — Mon, 01 Jan 2018 00:00:00 +0000

It is 2018 so it must be time to think about what my plans and goals are for the new year.

Buy a house

This a huge goal for me. For many years I was content with renting and thought owning my own home wasn’t for me, but then I went and started a family and my career started going in the right direction and my thoughts about this have changed.

It is going to take a lot of work to make this goal happen in 2018. First I need to save enough money for the deposit, which is far from easy with a wife and two children. Second I need to get our current place sorted ready to physically move which again won’t be easy. Third I need to make a decision which will affect where I live for the next 25 years. At the moment I am torn between the size of property and its physical location, I need to balance both of these and meet the requirements from the rest of the family.

I do think this is an achievable goal so wish me luck.

Progress in my career

I need to be careful how I phrase this in case past, current or future employers are reading. I have been working hard at my current role for over a year and I am starting to wonder what is next for me. I am not entirely sure what form this is going to take yet, I have a few ideas on the go and hopefully I can reveal more soon.

Another one that is very achievable, as long as I have it in the back of my mind as the year goes on progress should be made.

See more of family

I recently missed a family celebration due to the expansion of my own family and it reminded me that there are aunts, uncles and cousins that I haven’t seen in years and it would be nice to reconnect. Also as my own boys get older and are learning to interact it would be great if they could get to know my own family better.

I don’t have a clear idea how to achieve this one as it takes two but I am going to make more of an effort.

Celebrate 5 years of marriage

Wow how has it been five years since we got married? At some point in 2018 I will take time out from the usual and spend it with the wife away from the boys. I won’t give away more details as I want that to be a surprise.

Celebrate the birth of my two sons

I don’t want to get my boys christened as I would like them to make up their own minds when they are old enough, however I do want to celebrate all that they are with a thanksgiving service. Not sure when we are going to sort this, however it is my hope that it will be soon.

Family holiday

Not much of a goal but in the summer I will take time out to spend a week away with the family. In previous years we have done scotland, wales and northumberland. This year I am thinking of somewhere not too far away as Edward is still small, but maybe near where I grew up.

Lightning Talk

A lightning talk is a very short talk or presentation given at a conference or user group. Friends of mine are having lots of success talking about what they know about and I think it might be time for me to have a go. I am not a natural public speaker so this is a step out of my comfort zone, however I am going to start small and see what happens. At the moment I don’t know what my subject or topic is going to be, my suspicion is that it will be devops related as that interest me more but watch this space.