Blog: Markov Wanderer

Author

Arthur Turrell

The false economy of bad IT

work chat
productivity
public sector
Many of us will have experienced bad hardware or software at work. Applications that freeze when you try and do something. A lag when typing. Some programmes ceasing to work…
17 Sep 24
6 min

TIL: how to create and work with a MySQL database on Azure

code
cloud
In this TIL, I find out how to create a new MySQL database on Microsoft Azure. This is a place to store structured, tabular data. Note that the instructions below assume you…
24 Apr 24
9 min

TIL: how to create and work with remote blob storage on Azure

code
cloud
In this TIL, I find out how to create a new blob storage account on Microsoft Azure. This is a place to store unstructured data of any kind (as opposed to, say, a SQL…
23 Apr 24
4 min

TIL: how to connect Visual Studio Code to Azure Virtual Machines

code
research
cloud
python
rstats
In a previous blog post, I looked at how to connect desktop-based Visual Studio Code to a Google Cloud Virtual machine; today, it’s how to do the same using a virtual…
08 Mar 24
6 min

TIL: how to create a virtual desktop from a cloud virtual machine

code
data science
research
Researchers frequently want to be able to access a second computer that works like a normal computer (think a virtual desktop rather than a virtual machine + command line)…
04 Dec 23
3 min

Data science with impact

code
python
rstats
data science
public sector
I was recently asked to give a talk at No. 10 Downing Street on the topic of data science with impact and, in this post, I’m going to share some of what I said in that talk.…
13 Nov 23
32 min

Why have a model registry?

code
open-source
cloud
data science
Many large institutions, including in the public sector, have a set of forecasts, predictions, or estimated statistical relationships (perhaps from a linear regression)…
13 Oct 23
12 min

Building an API in the cloud in fewer than 200 lines of code

code
open-source
cloud
data science
python
Cloud tools and Python packages have become so powerful that you can build a (scalable) cloud-based API in fewer than 200 lines of code. In this blog post, you’ll see how to…
28 Sep 23
15 min

The explosion in time series forecasting packages in data science

code
data science
python
time series
There have been a series of sometimes jaw-dropping developments in data science in the last few years, with large language models by far the most prominent (and with good…
12 Sep 23
8 min

TIL: Obsidian, and integrating it with Zotero

writing
research
TIL
I’ve long been interested in how best to store knowledge; so much that I wrote about it in this post (in the context of the public sector). Today I learned how to combine…
09 Jul 23
11 min

The self-storage problem meets chatGPT

code
data
analysis
llm
In a previous post, I looked at four ways we might be able to establish the way that the number of self-storage facilities is trending over time. You can read that post using …
30 Jun 23
8 min

The mystery of stuff: why all the self-storage?

code
data
geospatial
analysis
llm
There’s a mystery at the fringes of our towns and cities: beyond the concrete circulars and just off the dual carriageways, a seemingly growing amount of our stuff is…
15 Jun 23
19 min

Data science maturity and the cloud

code
python
rstats
data science
cloud
work chat
public sector
Data science has enormous potential to do good in the public sector. The efficiencies that are possible from automation and reproducible analytical pipelines alone are…
01 Mar 23
12 min

The public sector could be better at managing knowledge ‘data’: what can we do?

economics
productivity
public sector
Who thinks the public sector is good enough at managing its stock of knowledge; the ideas, strategies, processes, and decisions that go into the efficient provision of…
08 Feb 23
19 min

Why you shouldn’t code on your work laptop

code
open-source
cloud
data science
work chat
“Nobody ever got fired for buying Microsoft” goes an old saying. Actually, it was probably first said in the 1980s in reference to IBM (School Microcomputing Bulletin 1983)…
13 Jan 23
10 min

In praise of APIs (application programming interfaces)

code
data
In this blog, I look at some of reasons why APIs are such a great way to share data.
18 Dec 22
7 min

TIL: How to break RSS feeds

blogging
code
writing
TIL
Note: this is the first post under a new tag called TIL or “today I learned”. These are shorter format posts that lower the barrier to blogging and capture a mini piece of…
02 Nov 22
2 min

Welcome To The New Home for My Blog

news
This is the first post on a brand new blog site: welcome!
16 Oct 22
1 min

Visual Studio Code on the Cloud

code
research
cloud
python
rstats
Visual Studio Code is incredibly powerful, whether it’s for writing markdown, writing quarto (.qmd) files, getting syntax highlighting and peerless language support (eg…
28 Sep 22
12 min

Writing a Research Blog Post

blogging
research
writing
In this post, you will find hints and tips for writing impactful blog posts that summarise research or analysis. This is a cross-post with a new page on Coding for Economists…
21 Feb 22
12 min

Setting up R in Visual Studio Code

code
rstats
This post will show you how to set up Visual Studio Code as an integrated development environment for the statistical language R. This will include some useful features such…
04 Nov 21
3 min

Three ways to blog with code

code
blogging
Typically, what I want to do when I create a blog post is to combine text, code, and code output, and then push it to the github repo that hosts my website. But what are the…
12 Feb 21
13 min

10 less well-known Python packages

code
open-source
python
(Remember that to use these, you will need to run pip install packagename on the command line.)
24 Oct 20
29 min

Get organised

code
data
research
This monster blog post is going to discuss how to organise your a data science project or research project: data, code and outputs. I’ll cover how to structure the project…
26 Jun 19
34 min

Specification curve analysis

code
research
open-source
econometrics
Since publishing this post, I have written the specification_curve package for Python. specification_curve automates some aspects of specification curve analysis, namely…
25 Jan 19
39 min

Putting women scientists onto Wikipedia

outreach
code
data
In a previous post, I shared links about the predictors for not participating in higher education, and about how it is difficult to reach audiences in “remote rural or…
25 Aug 18
7 min

Who is not participating in Higher Education?

outreach
data
Given my work in both economics and Science, Technology, Engineering, and Mathematics (STEM), I’ve become interested in what factors determine groups’ participation in…
18 Aug 18
5 min

Why the latest, most exciting thing in machine learning is… game theory

research
machine-learning
And when I say latest, this particular method was invented in 1953.
11 Jul 18
4 min

Econometrics in Python Part IV - Running many regressions alongside pandas

code
econometrics
python
The fourth in the series of posts covering econometrics in Python. This time: automating the boring business of running multiple regressions on columns in a pandas dataframe.
05 May 18
46 min

Econometrics in Python part III - Estimating heterogeneous treatment effects using random forests

code
econometrics
machine-learning
python
The third in a series of posts covering econometrics in Python. Here I look at ‘causal forests’.
28 Mar 18
6 min

Econometrics in Python Part II - Fixed effects

code
econometrics
python
In this second in a series on econometrics in Python, I’ll look at how to implement fixed effects.
20 Feb 18
20 min

Econometrics in Python part I - Double machine learning

code
econometrics
python
The idea is that this will be the first in a series of posts covering econometrics in Python.
10 Feb 18
6 min

Making a publication quality plot with Python (and latex)

research
visualisation
code
python
I now recommend the style file below for quick, publication quality plots in Python using Matplotlib (tested on 3.3.4 and Python 3.8). To use the style, save it in a file…
31 Jan 18
10 min

The ONS API

data
code
python
The Office for National Statistics (ONS) produces most of the macroeconomic statistics for the UK. I was delighted to discover recently that they had been working on an API.
28 Dec 17
2 min

A better narrative on narratives

research
news
Pigou, Keynes and Shiller all recognised the importance of narratives and sentiment for the economy. But we don’t know too much about how narratives spread. One of the most…
10 Nov 17
3 min
No matching items