2017-06 on Funky Si's Blog (Dev)

Using Azure WebJobs to automate

funkysi1701@gmail.com (funkysi1701) — Mon, 26 Jun 2017 20:00:45 +0000

I keep hearing about Azure WebJobs but I have never used them. Time to change this.

WebJobs are a feature of Azure App Service that can run a script at a specific time. In my case I would like to hit a specific url of my website at the same time every day.

To the right you can see an example of the WebJobs form on the Azure portal that you need to fill in.

You need to supply a name for your webjob.

You need to upload the script that will run in my case I used a powershell script. My script consisted of which basically just loads the url specified.

$progressPreference = "silentlyContinue";
$result = Invoke-WebRequest -Uri ("https://www.google.com") -Method Get -UseBasicParsing;

Type refers to if your job will be triggered or run continuously, I want it to be triggered.

Triggers refers to if you want it to be scheduled or manual, something that you can run on an ad hoc basis. I of course want scheduled.

If you are familiar with the linux CRON then the next box will make sense to you for everyone else I will try and make sense of it. The box consists of 6 numbers which can either have a value or a *. The numbers correspond to the following {second} {minute} {hour} {day} {month} {day of the week}.

A hourly job would be expressed as 0 0 * * * *, ie every day of week, every month, every day, every hour and only when minute and second equals zero. For more help with this check out the MSDN docs about it. I want to use 0 30 21 * * * to run daily at 9.30pm.

That’s it everything setup, now time to wait and see if it works.

Oh no!

It failed to run at the specified time.

The reason for this is the scheduler requires the feature Always On to be turned on which is not available in the free App Service. Before you reach for your wallets, I found a solution on this blog post that allows them to run on the free tier.

The thinking behind this solution is you need to keep the website alive throughout the day so Tom has created a script that does this. His script can be found on his blog or on his github page .

Set this script up to run every 5 minutes (0 */5 * * * *) like the example above.

The nextthing you need to do is create a Custom connection string in the Application Settings blade called SecretThing. Tom’s script references this to access the website and keep it alive. The password you need to put in SecretThing can be found in you publish profile (downloaded from the Overview blade in the Azure portal). For more details and a better explanation check out Tom’s blog .

One last thing to mention about WebJobs is that you can see details about when they have run at https://[YourWebAppName].scm.azurewebsites.net/azurejobs/#/jobs and this can be a great place to help debug your scripts.

Fiddler and APIs

funkysi1701@gmail.com (funkysi1701) — Mon, 19 Jun 2017 20:00:45 +0000

A while ago I blogged about promoting my blog with Buffer. At the time I made use of the nuget package BufferAPI but lets look at some improvements I can make.

The BufferAPI package worked great from my console app, but when I tried to use it from a Controller in an MVC app I never got it to work. Lets look at the API docs and see if I can rewrite it.

There are two main types of API calls GET which gets data from the server and POST which posts data to the server. These come from the types of HTTP requests.

I quickly figured out how to use the GET API call to authenticate using https://api.bufferapp.com/1/profiles.json?access_token=XXXX

However POST was defeating me. That was until I remembered Fiddler .

I had heard Troy Hunt (and others) talk of using Fiddler to examine what data is being passed among websites. Troy uses it to do a man in the middle test to see what information can be stolen.

It is really easy to setup, install Fiddler, click yes to a few security warnings and you can see what information is being passed from your code to remote APIs.

Once I had Fiddler installed I could compare what information is being passed between a successful API call using the BufferAPI nuget package and an unsuccessful API call using my code.

Fiddler also showed that passing my authentication token in a POST request is much better. Despite both GET and POST being encrypted when using HTTPS, anything at either end that logs URLs will have a log of your username and password.

If you have not tried Fiddler, give it a try especially if you are doing things with API calls.

Add Power BI data to a webpage

funkysi1701@gmail.com (funkysi1701) — Mon, 12 Jun 2017 20:00:45 +0000

Last week I talked about Power BI , what it is and some of the different services you can use with it. This week lets add some of that data to a simple web page.

For this example I am going to add the google analytics data from this website to this page.

First login to your Power BI https://app.powerbi.com/

Click the get data link at the bottom left.

Click My Organisation to bring up the app search box.

Click the Apps tab and search for “google” in the search box, you should then see Google Analytics, click into this and then click the get it now button.

Log into your google account. If you have multiple google accounts I found it worked best to sign out of all of them or run this in an incognito window.

Once you are signed in you should see a list of the different google analytics data you have, select the one you want to use and click import.

Power BI will then go away and start loading the data.

Once loaded go to Reports and select the Google Analytics that has been loaded. If you have more than one, it is a good idea to rename each one eg Corporate Site Google Analytics, Blog Google Analytics so you won’t get mixed up.

In the file menu select Publish to web and agree that you are OK for this to be made public.

You will then be given a piece of HTML code that starts with <iframe copy this onto your web page. Reload your webpage and you should see something similar to mine below.

It should be noted that while you can but the webpage containing the iframe behind a login page, the data could still be accessed if you knew the url contained within the iframe, this is why the link can be emailed and continue to work.

What is Business Intelligence here?

funkysi1701@gmail.com (funkysi1701) — Mon, 05 Jun 2017 20:00:45 +0000

The term Business Intelligence (BI) refers to technologies, applications and practices for the collection, integration, analysis, and presentation of business information. The purpose of Business Intelligence is to support better business decision making. Essentially, Business Intelligence systems are data-driven Decision Support Systems (DSS). Business Intelligence is sometimes used interchangeably with briefing books, report and query tools and executive information systems.

What is Power BI?

Power BI is a business analytics service provided by Microsoft. It provides interactive visualisations with self-service business intelligence capabilities, where end users can create reports and dashboards by themselves, without having to depend on any information technology staff or database administrator.

Microsoft describe it as follows: Power BI is a suite of business analytics tools that deliver insights throughout your organisation. Connect to hundreds of data sources, simplify data prep, and drive ad hoc analysis. Produce beautiful reports, then publish them for your organisation to consume on the web and across mobile devices. Everyone can create personalised dashboards with a unique, 360-degree view of their business. And scale across the enterprise, with governance and security built-in.

What can you do with it?

One of the first thing you can do with Power Bi is connect some of the apps that you use.

To sign up to use Power BI go to the Power BI website .

Once you are signed in click the get data link in the bottom left corner and browse through the apps you can use.

If your website uses google analytics you can connect it up to Power BI and see some cool analytics. The below map was generated from some google analytics data, you can see that this website has had users across the globe, with UK and USA getting a lot of hits.

Another thing you can do is connect Azure Application Insights. This provides some similar data to google analytics about who is visiting your site.

If your code is hosted in Visual Studio Team Services you can get some cool analytics from your commit history.

There is a huge long list of apps you can connect to power BI so you can start getting all sorts of analysis very easily. I am hopefully going to talk another time about getting custom data into Power BI.