Tag Archives: data
Remove CSV text qualifiers within a field using Python
Have you ever tried to import a CSV into a database and found that it won’t load because one of your CSV fields has a text qualifier inside the field itself? In this post I look at how to resolve this issue using a few lines of Python code.
Memory: how a computer stores and accesses information
How does a computer ‘remember’ something, and furthermore how is it able to recall some things but not others? In this post, we will explore the concept of memory, what it means and how a computer uses it.
How does a computer store data: dates
In this post, we will examine the concept of dates and time to see how it is possible to store them in computer memory.
How does a computer store data: text
In this post, we will look at how a computer is able to store and represent text – once again, using just ones and zeros.
How does a computer store data: numbers
We will examine exactly how data is stored by a computer. In this post, we will look at numbers.
My Top 10 ETL Best Practices
It’s been over ten years since I coded my first ever ETL routine. Since then I’ve collected together a number of important lessons and best practices, presented here as my very own ‘Ten Commandments’ of ETL. 1. Know your data Whether you’re defining an ETL strategy, designing a set of data flows or writing the code, the single […]
Undermining intuition
An interesting piece in today’s Telegraph from Tom Chivers highlights the disconnect between popular perception and harsh statistical reality. UK crime rates are demonstrably going down, but the majority of people in the UK believe crime to be on the rise. The UK’s foreign aid budget is a small fraction of overall expenditure on pensions and education, […]
Social Media + Big Data = Confused Journalism
I’ve just come across this article, which claims that the combination of social media and big data will be responsible for who wins the US Presidential election in 2016. It’s a bold assertion, and it comes on the back of many other opinion pieces that vaunt the major role of big data in the 2012 election. […]
Busy Intelligence 