2019 on Funky Si's Blog (Dev)

Picard is back!

funkysi1701@gmail.com (funkysi1701) — Thu, 08 Aug 2019 20:00:45 +0000

It has just been announced at the annual Star Trek Las Vegas (#STLV ) convention that Patrick Stewart is going to return to his role as Jean-Luc Picard in a new TV Show.

Few other details have been announced other than the TV show will take place 20 years after the film Star Trek: Nemesis and will feature the character of Jean-Luc Picard.

Lets run though a few things I would like to see from this new show.

Low on Action

Star Trek: Discovery is a great show and has brought Star Trek back to our TV shows. However it is an action show, it zooms along so fast and with so much action I would appreciate a show that can takes its time a bit more.

I want a show that can show Picard carefully considering some moral issue and making a decision. Something we were used to seeing on TNG.

Feature Picard’s excellent speeches

One of the best features I can think of Jean-Luc Picard is his ability to make speeches about one issue or other.

Lets look at a few examples:

Insurection : Picard speaks about the forced relocation of a group of people.

How many people does it take, Admiral, before it becomes wrong? Hmm? A thousand, fifty thousand, a million? How many people does it take, Admiral?

First Contact : Picard is hell bent on revenge for what the Borg have done to him and refuses to destroy the Enterprise to save his crew.

“I will not sacrifice the Enterprise. We’ve made too many compromises already. Too many retreats. They invade our space and we fall back. They assimilate entire worlds, and we fall back. Not again! The line must be drawn here, …this far, no further! And I will make them pay for what they’ve done.”

Measure of a Man : Picard argues for the rights of Data.

“You see, he’s met two of your three criteria for sentience, so what if he meets the third. Consciousness in even the smallest degree. What is he then? I don’t know. Do you? (to Riker) Do you? (to Phillipa) Do you? Well, that’s the question you have to answer. Your Honour, the courtroom is a crucible. In it we burn away irrelevancies until we are left with a pure product, the truth for all time. Now, sooner or later, this man or others like him will succeed in replicating Commander Data. And the decision you reach here today will determine how we will regard this creation of our genius. It will reveal the kind of a people we are, what he is destined to be. It will reach far beyond this courtroom and this one android. It could significantly redefine the boundaries of personal liberty and freedom, expanding them for some, savagely curtailing them for others. Are you prepared to condemn him and all who come after him to servitude and slavery? Your Honour, Starfleet was founded to seek out new life. Well, there it sits. Waiting. You wanted a chance to make law. Well, here it is. Make a good one.”

Ask important questions

We live in uncertain times. There are lots of pressures on today’s world and we need to acknowledge these. Picard is an excellent character to use to ask important questions of the day.

What does it mean to be human?
How should we treat people that are different to us?
How should we react to extremism?

Deal with his age

In the last episode of TNG a future version of Picard is shown 25 years into the future. This is a similar time period so it will be interesting to compare the two. Have Picard look back on his life and ask questions about his life choices.

Be consistent with his past

Picard is a well known character. So writers please don’t change him. I want a character we know, not a character that looks like Picard but acts in a way contrary to what we would expect.

Picard has a history we know, he served on the stargazer and of course the Enterprise, he has an artificial heart, he was assimiliated by the Borg. All

Reference 50 years of Star Trek

Discovery and Enterprise are TV shows which have a problem. There is Trek going on after them which imposes a few rules on what can and can’t be done.

This is a TV Show that will take place after every other Trek TV and Film and can do whatever the creators want. This gives freedom, but it also gives the responsibility to reference what has gone before.

The 2009 Star Trek film features the destruction of Romulus and Spock travelling back in time. How about Picard dealing with surviving Romulans helping rebuild their society, continuing the work that Spock started in the TNG episode Unification.

I look forward to finding out what the Star Trek production team has in store for us. If they have Patrick Stewart convinced I am sure it is going to be good.

Did you know you can track episodes from TNG and Picard in Episode Atlas .

Create Azure resources via Terraform

funkysi1701@gmail.com (funkysi1701) — Sat, 11 May 2019 20:00:45 +0000

Terraform is a way to script the creation of Azure Resources. By scripting the creation of your resources you can run them again and again and be certain that everything is configured exactly the same as the script has specified. No more forgetting to tick the tick box, if it’s in your script then it will be deployed with exactly the same configuration.

The process of executing a terraform script follows this basic process.

terraform init – Initialises the terraform directory terraform plan – generates and displays what will be executed terraform apply – applies the terraform script to your Azure subscription terraform destroy – deletes your Azure resource that were created in the previous step

Lets look at how you might execute a terraform script from your build or release pipeline.

I am executing my terraform scripts from my release pipeline and as I don’t want to check binaries into my source code I am going to add a bit of plumbing to get things in the right places. The terraform binary can be obtained from https://www.terraform.io/downloads.html .

To start with I am executing a Powershell script to download the terraform binary, this is in the form of a zip file, so i need to download and unzip it.

#Download
$url = "https://releases.hashicorp.com/terraform/0.11.13/terraform_0.11.13_windows_amd64.zip"
$output = "terraform.zip"
(New-Object System.Net.WebClient).DownloadFile($url, $output)

#Unzip
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Compression.ZipFile]::ExtractToDirectory($output, Get-Location)

Now that I have the terraform binaries I can just run Azure Cli and execute the terraform commands above.

Now it is time to look at the the terraform script. This particular script creates a resource group, an app service plan and a webapp.

# Configure the Microsoft Azure Provider
provider "azurerm" {
subscription_id = ""
client_id = ""
client_secret = ""
tenant_id = ""
}
resource "azurerm_resource_group" "rg" {
name = "resource-group"
location = "northeurope"
}
resource "azurerm_app_service_plan" "plan" {
name = "planname"
location = "${azurerm_resource_group.rg.location}"
resource_group_name = "${azurerm_resource_group.rg.name}"
sku {
tier = "Standard"
size = "S1"
}
}
resource "azurerm_app_service" "webapp" {
name = "example-app-service"
location = "${azurerm_resource_group.rg.location}"
resource_group_name = "${azurerm_resource_group.rg.name}"
app_service_plan_id = "${azurerm_app_service_plan.plan.id}"
}

There are four guids that you need to fill in from your azure subscription and to find these you need to run a few commands from Azure Cli. Check out https://learn.microsoft.com/en-us/azure/virtual-machines/linux/terraform-install-configure for more detail on these commands.

Run these either from Azure Cloud Shell or from Powershell command prompt after Azure Cli has been installed.

az login – to login to azure a browser window will open up.

az account show –query “{subscriptionId:id, tenantId:tenantId}” This will show you your subscription id.

az account set –subscription="${SUBSCRIPTION_ID}" Where ${SUBSCRIPTION_ID} is the subscription id you found in the last step.

az ad sp create-for-rbac –role=“Contributor” –scopes="/subscriptions/${SUBSCRIPTION_ID}" Where ${SUBSCRIPTION_ID} is the subscription id you found in the last step and assumes you have permission to the subscription.

This last command will about a series of Guids which you need to fill in at the top of your terraform file appId = client_id, password = client_secret, tenant = tenant_id and subscription_id you already have.

This just gives you a taste of what you can do with terraform. Looking at the Azure Terraform docs there are loads of different resources you can create and even a github repo with some samples.

Gated Release

funkysi1701@gmail.com (funkysi1701) — Fri, 05 Apr 2019 20:00:45 +0000

Automated releases of software are great but how can we add an element of feedback so only good releases go live.

I have been using Azure DevOps to release my PwnedPass android app to the Google Play Store for a while now. There are options to deploy to the alpha, Beta or Production tracks and even to set % of users to target. For the full range of options check out the Google Play extension for Azure DevOps.

My release starts by publishing to 10% of users on the production track, my next step makes use of the increase rollout option to increase this %, you can have as many of these additional steps as you want until you reach 100% of your users.

Now if you run this release now it will just run through each of the steps one after the other. Now of course you can add a pre or post approval to your pipeline but this just adds a manual dependency to your release. Whoever does the approving needs to check things are working before approving or worse just approves regardless.

Azure DevOps has the concept of gated releases which allows you to add automated checks before or after a release happens. These automated checks can be any of the following:

An Azure Function
A Rest API call
Azure Monitor Alert
Query Work Items
Security and Compliance Assessment

We are going to make use of the Azure Monitor Alert, to create an alert from your Application Insights data and only continue the rollout if no failures are detected.

Open up your application insights resource in the Azure portal and look in alerts. Click add new alert rule.

Select your application insights resource in Resource, In Condition choose a condition to check, I chose Failed Requests, so every time a failure is registered in my API I can stop the deployment. The exact criteria you want to use is entirely up to you.

Create an action group, I just set my alert to send an email to myself but there are other alert actions you may want to try. Give your alert a name and description and click save.

Now all we need to do is make Azure DevOps make use of this alert. In your release pipeline select the pre-deployment conditions of your second step and open up the Gates section.

Choose a suitable time to evaluate, I have been using something long like 12 or 24 hours so if there are problems there is time for it to be noticed. Choose Version 1 of the task (I was not able to get it to work with Version 0)

Now select your Azure subscription and Resource Group and leave the rest of the settings as they are. Now your Deployment will stop and analyse application insights for any Failed requests and will halt if it finds any.

I am still testing this out but it will take a few days to figure out if this what I want due to the large time scales involved. I feel this is going to be an improvement of manually approving release steps.

Documenting your API

funkysi1701@gmail.com (funkysi1701) — Wed, 27 Mar 2019 20:00:45 +0000

So you have created a super API that does something amazing. How do you document it so people will use it?

One way of easily documenting your API is to install the Swashbuckle package.

Install-Package Swashbuckle.AspNetCore
Install-Package Swashbuckle.AspNetCore.Swagger

Then in you startup.cs add the following lines

//In ConfigureServices

services.AddSwaggerGen(c =>
{
 c.SwaggerDoc("v1", new Info { Title = "API", Version = "v1", Description = "An API Description" });
 c.IncludeXmlComments(string.Format(@"{0}\API.xml", System.AppDomain.CurrentDomain.BaseDirectory));
});

//In Configure

app.UseSwagger();

app.UseSwaggerUI(c =>
{
 c.SwaggerEndpoint("/swagger/v1/swagger.json", "API V1");
 c.RoutePrefix = string.Empty;
});

Now when you browse to your API you will see the swagger documentation system.

The RoutePrefix setting controls the path in which swagger will display. I have my docs at the root, but you might want them under the /docs or similar path.

The IncludeXmlComments setting from the ConfigureServices method allows you to load in any XML comments you have added to methods. For this to work you need to enable a setting to your build.

The XML documentation file must be ticked and contain a path. Everytime you do a build, a XML file will be generated which contains all the comment blocks you have added to your code.

Swagger will then use this XML documentation file to produce lovely looking documentation without you having to do anything extra.

Azure Key Vault

funkysi1701@gmail.com (funkysi1701) — Tue, 19 Mar 2019 20:00:45 +0000

Azure Key Vault is a secure way of storing your keys, certificates and secrets so your application can access everything it needs to but you don’t have them being stored insecurely anywhere such as in source control.

I have been wanting to give Azure Key Vault a try for a while now as it can make use of Azure Active Directory to give your web app an identity so it can authenticate itself into the key vault to access secrets. Pretty clever but with a lot of moving parts a bit complex.

For my example I am just going to connect to my Key Vault and get a secret and display it somewhere on a web page. This is of course not what you want to do as secrets are secret and shouldn’t be displayed just used to authenticate into whatever, however it is an easy way to prove I am connecting to the Key Vault and everything is working.

Lets look at some code. I have a .net core application and to start with lets install three nuget packages.

Microsoft.Azure.KeyVault
Microsoft.Azure.Services.AppAuthentication
Microsoft.Extensions.Configuration.AzureKeyVault

I’ve not include version numbers as these will no doubt get updated over time but hopefully it will still work.

Now in your Program.cs add the following code, replacing [KeyVaultName] with the name of your Key Vault.

 public class Program
 {
 public static void Main(string[] args)
 {
 CreateWebHostBuilder(args).Build().Run();
 }

 public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
 WebHost.CreateDefaultBuilder(args)
 .ConfigureAppConfiguration((context, config) =>
 {
 var builtConfig = config.Build();

 var azureServiceTokenProvider = new AzureServiceTokenProvider();
 var keyVaultClient = new KeyVaultClient(
 new KeyVaultClient.AuthenticationCallback(
 azureServiceTokenProvider.KeyVaultTokenCallback));

 config.AddAzureKeyVault(
 $"https://[KeyVaultName].vault.azure.net/",
 keyVaultClient,
 new DefaultKeyVaultSecretManager());
 })
 .UseApplicationInsights()
 .UseStartup<Startup>();
 }

Now all you need to do is look at your configuration to pull out secrets from your Azure Key Vault. If you have a secret called AppSecret then you can use the following code snippet to retrieve its value, assuming _configuration is an implementation of Microsoft.Extensions.Configuration.IConfiguration.

_configuration["AppSecret"];

Now if you do all of this and run from an Azure Web App or run locally it will fail to pull anything from the Key Vault. You need to give your web app an identity and configure your key vault to allow access from that identity.

Once my code has been deployed to an Azure Web App I get the following error.

Lets look at fixing that, first lets give my web app an Identity. Open up the Azure portal and find the identity section of your web app and turn the setting on.

Now you need to grant that identity permission to your key vault. In the portal open up Access Policies in your key vault and click add Policy, select the identity of your web app in the principal box and select the following settings to grant access to your secret.

Now you have a website that can pull secrets out of Key Vault but only that unique identity. Anyone who has access to your source code will not have access to your secrets, even if they push your code to a different Azure Web App.

Tech I want to learn more about

funkysi1701@gmail.com (funkysi1701) — Tue, 05 Mar 2019 20:00:45 +0000

While at Microsoft Ignite I heard about a lot of cool tech that I want to know more about. The best way to learn something is use it to solve a problem.

So what can I build that is both useful and will let me play with some new tech?

I have a Xamarin Forms app Pwned Pass that has over 500 downloads on Google Play and over 80 downloads on the Microsoft Store. This has given me a small user base that I can use to make use of whatever I build.

My app makes use of the HIBP API created by Troy Hunt. I am going to build my own API, initially it will just make calls to the HIBP API. Building this will give me experience of building something with .net Core from design to deployment. I have made a start already on doing this, I have an empty .net core API project which deploys to an Azure web app using the build and release pipelines from Azure DevOps.

You may be wondering why I am not making use of Azure Functions to build this API. Azure functions is certainly a great technology that is worth learning about. However I have done a little bit with them in the past and I don’t believe I would be able to learn all the things I want to if I used Azure Functions. My primary goal is learning and sharing that learning via this blog. It may well be I move to using Azure Functions later on.

Another tech I am keen to learn more about is Azure Key Vault . This is a technology that allows the securing of keys, connection strings and certificates. I want my app to securely get keys and security information without any of it having to be committed to source code or shared insecurely.

Monitoring my app is also a key learning from me. I use application insights already, but I would like to extend my understanding of this so telemetry can be fed back into the build and bad deployments stopped.

Below is my complete list of learning and tech I want to touch on. It is a long list and I imagine it will get longer as I work through it. I want to regularly blog and share what I have been working on. I currently have a working build and release pipeline but nothing of note to build or release. I know Key Vault needs looking at early as the identity of the website in Azure is key to getting that tech working correctly.

Build API with .net core
Add build and release pipeline
Make use of Azure KeyVault for secrets, connection strings etc
Plugin My Xamarin app to make use of it
Monitor my API with Application Insights
Secure it with CSPs and log this into Report URI
Consider building a web frontend to my API using a javascript library or framework. Maybe react but this can be decided later.
Dockerize the API and add the creation of docker images to the build/release pipeline

Microsoft Ignite | The Tour – London

funkysi1701@gmail.com (funkysi1701) — Tue, 26 Feb 2019 20:00:45 +0000

I have just spent the first day at the conference Microsoft Ignite | The Tour. The conference was free I just needed to arrange travel and accommodation. Microsoft have really looked after me and all the other guests, breakfast and lunch has been provided, I got a free t-shirt, coffee (or tea) all day plus a beer or glass of wine to end the day. And that’s before you collect any free stuff the vendors are giving away.

The first session was Designing resilient cloud applications with Matt Soucoup and talked about some cloud technologies like Azure key vault and serving static files from blob storage. Unfortunately this had a few technical issues with the demos. I think it was just connectivity with the MongoDB backend but this slightly spoiled the session. As this was the only technical problem I noticed all day I can let it pass.

Next was a session on Azure DevOps mainly build and release pipelines called Deploying your application faster and safer with Brian Benz . A lot of this I knew but good to reinforce I am doing things correctly.

Next was a session on Application Insights called Detecting application anomalies with Telemetry with Matt Soucoup.

Probably the most useful session was on Docker and Kubernetes called Integrate containers and Kubernetes into your Azure DevOps build and release model with Marco De Sanctis . Going to spend some time looking through the examples from this session.

A session on Serverless covered Azure Functions, Azure Logic apps and the other Azure Serverless offerings. Investing in Serverless: less servers, more code with Simona Cotin .

Lastly was a panel discussion on the changes facing IT Pros and SysAdmins. What is the future of the IT Pro in a DevOps & Cloudy world? with Jennifer Stirrup , Baki Onur Okutucu , Amy Boyd and Stephen Thair . Should they learn to code, how should they make sure they keep up.

Tomorrow I have a loads more sessions, including ones about mental health, Azure pipelines, more Kubernetes stuff and dealing with failure.

Yaml Builds on Azure DevOps

funkysi1701@gmail.com (funkysi1701) — Thu, 31 Jan 2019 20:00:45 +0000

I have been using Azure DevOps (Or VSTS or VSO etc) for a while now and one of the great features is doing automatic builds with every check-in. This is more commonly known as a CI (continuous integration) build.

More recently I have started playing about with creating my build using YAML files instead of using the web user interface to create my build.

Why?

You may wonder why go to the effort of learning the YAML syntax when you can just create the build in Azure DevOps and then forget about it.

Mostly it is because the build changes over time and you shouldn’t just forget about it. If something changes over time then you might want to version control it, or look at a previous version.

Lets say you create a pull request that replaces a .net 4.7 web service with a .net core web service. If you have a CI build this PR will fail because it won’t build. If you change the build first any other builds going on will fail. What you want in this case is the build to be associated with that branch or PR. Any builds before you merge this change in will continue to work and any after this change will also work.

How?

How do you get started with YAML builds? Well the first thing is to make sure that YAML builds are turned on, as I write this I believe they are still a feature you can turn on or off. Have a look in Preview features and make sure they are turned on.

Next look at any of your existing builds and click the View YAML link. This will show you an example YAML file of your existing build. You could just save this as azure-pipelines.yml and checkin to the root of your project. You can also click the add new build pipeline option, this will give you some templates to start you off.

The YAML file consists of a series of build steps usually called tasks, with a few settings before to configure things like parameters or build agents. Detailed docs about the syntax of the file can be found here .

For my mobile app my YAML files consist of downloading nuget packages, building the solution, building specific projects with desired settings, running powershell or other scripts to set things up and finally publishing the results of the build as artifacts so that they can be used in any releases.

Secure It!

Avoid committing passwords and secure keys into source control. I have found you can upload secure files via Azure DevOps and then add a download secure files step at the start of your build. This allows the secure file to be used during the build but the contents of the file can’t be viewed by anyone with access to the source code.

I find it often takes a bit of thinking about how to achieve this, but it is usually possible to keep keys and secrets secure.

Pwned Pass Update

funkysi1701@gmail.com (funkysi1701) — Wed, 23 Jan 2019 20:00:45 +0000

Its been a while since I first released Pwned Pass so lets have a look at where we are now.

We are very close to 500 Downloads from Google Play and we have recently smashed past 100 active installs, peaking at 116 and even now we are still over 100. I have had 9 reviews (6 x 5*, 2 x 1 * and a 4 *) which averages out at 4 * Over Christmas I released a UWP version that can be found in the Microsoft Store. This has currently had 9 downloads and even had a download to Windows Mobile (someone out there still likes the platform!) I have a fairly smooth deployment process using Azure DevOps. After every check in of code a build runs which compiles the UWP and Android versions. The build also increments the version numbers that is required to deploy to either of the app stores.

Every successful build of the master branch will kick off a release to the Beta track of Google Play. If I am happy I then release to 10% of the Production track, which can then be increased to 100% (or halted). The release to Microsoft Store happens after the Beta track of Google Play. Only reason for this order is that there isn’t a beta area for UWP apps so I want to quickly test change on android before rolling out for windows.

All these steps require confirmation by me before proceeding and often don’t get further than the beta track.

A further development is that I have open sourced the source code to github do take a look if you are curious or want to contribute. With the purchase by Microsoft there are easy ways to connect github repositories to Azure DevOps. Once I create a Pull Request in github it creates a build in Azure DevOps and all the build and release steps can happen.

I am still not 100% sure if I want to keep my bug and issue tracking in github or Azure DevOps as both have features for doing so.

One future improvement I want to make is to automate the creation of screenshots. When I create a new feature and it gets checked in. I would like to automatically created screenshots of the key pages and submit them to the different app stores. Currently I am not sure if this is possible or how to go about it. I have some ideas to experiment with so we will see what I can do.

What I planned to achieve in 2019

funkysi1701@gmail.com (funkysi1701) — Tue, 01 Jan 2019 20:00:45 +0000

This is my annual lets make some goals for the new year blog post. So in no particular order.

Improve House

I can’t outdo last years goal of buying a house but I can continue to make efforts to improve it. I was going to put something about decorating or doing something to our bedrooms, however today I spotted a hole in the roof so this may well absorb most of my home improvement budget.

Connect with local area

Since we moved in October we haven’t really connected with our local area yet. My only interaction with my neighbours was when my car blocked his drive!

I am going to make efforts to change this in 2019. I don’t exactly know how yet, but I have some ideas. My son starts nursery and later school this year so could be one way to meet other parents and teachers. I want to connect with a local church, so far this hasn’t happened so need to make more effort with this. There are other local groups that put on child friendly events which would be good to attend.

Lightning Talk

I failed last year to do this so it is back on my list. I know I must have stuff of value to share so going to try again on this.

A lightning talk is a very short talk or presentation given at a conference or user group. I am not a natural public speaker so this is a step out of my comfort zone, however I am going to start small and see what happens.

##Family Holiday As always we Fosters are going to have a holiday in the summer and have some quality family time. I also want to take the boys to London for a weekend.

Blogging

This blog has taken a bit of a backseat in 2018 so I want to try and refocus it for 2019. My commitment is for 12 blog posts in 2019, that is one a month. It doesn’t sound like much but I know how easily life can get in the way.

Having said that my log has had record views this year. 7159 views. However I believe this number to be incorrect and has been inflated by my app development and some of the automated processes taking place.

Routine

Last goal is to try a get more of a routine going. With two small boys, one of which is starting school in September a routine is going to be essential and will allow all of us all a chance to get more things done.