Blog

A data science blog for Petroleum Engineering.

Topics covered

Subject ▸ data science

One of the most challenging things in dealing with data is Nested Structures. In a perfect world, data would be tables (rectangular) and be tidy. If physicists are finding the right format, they should also work for petroleum engineering folks.

The image is a screenshot of Fig.1 from the paper [“Machine Learning in High Energy Physics Community White Paper”] (http://arxiv.org/abs/1807.02876)

Link to post in Linkedin

The cloud and the sarcam of AWS easy as pie

28 December 2018

data science, cloud

Read from an article yesterday how learning the Cloud is a must if you are in data science and machine learning. I made some annotations: Although the article is a discussion on DevOps, there are parts that go beyond and touch data science and machine learning interests. That was just in case you thought you had it hard learning Python or R. The article is named How To Become a DevOps Engineer In Six Months or Less.

How can a Petroleum Engineer kick start with Data Science?

4 December 2018

data science

I would start by identifying acute problems in your area of expertise (domain): production , reservoir, drilling, completions, geophysics, chemistry, seismic, geophysics, etc., that you feel could be resolved by applying data science. They may be big problems or small ones. Start with the small ones, or break the big ones in manageable pieces that you can address one step at a time. Once you have two or three data science “project” candidates, start applying the basics to solve the problem.

Data Science for Petroleum Engineering: How does someone become good at Deep Learning?

23 September 2017

machine learning, data science

I watched few days ago the interview from professor Andrew Ng to one of the luminaries of deep learning and artificial intelligence, Dr. Youshua Bengio. He has written books and dozens of papers on deep learning and neural networks. I liked the style. Pretty down to earth stuff. Just the way professor Andrew likes to do: bringing machine learning, deep learning to the masses. So the question remains: do petroleum engineers need to learn data science, computer science, statistics, machine learning, neural networks, virtualization and GPU based engineering?

Data Science for Petroleum Engineering. Part 6 - Tidy data

13 September 2017

data structures, data science

One of the first concepts that one learns when working with data is rearranging raw data into tidy datasets. A tidy dataset not only means having the data in a row-column format but in such a way that a row corresponds to an observation and a column to a variable. This facilitates enormously the analysis. I know this could sound a little bit confusing, so I will show what raw data and tidy data looks like with an example.

Transforming Petroleum Engineers in Data Science Wizards

25 August 2017

data science, petroleum engineering

Once in a while I get messages from colleagues asking for tips on Data Science applied to Petroleum Engineering. This is stuff I have collected over time (responses), advice to follow to become a Petroleum Engineer and Data Science wizard: Complete any of the Python or R online courses on Data Science. My favorites are the ones from Johns Hopkins in Coursera, complementing with DataCamp short workshops. Just two that come quick to my mind.

Data Science for Petroleum Engineering - Part 5.3 Finding and filling missing well data in alphanumerics

24 August 2017

data science, production technology

This is what we will reviewing in this lecture. NOTE. You can find the PDF version of the R markdown notebook in GitHub at this link. The reproducible R markdown notebook itself is here. Both are full versions of this LinkedIn article. For the time being, LinkedIn publishing does not support markdown which would make sharing scientific and engineering documents much easier. Load the raw data file # code We will see that some well names can be fixed manually and others should be done automatically with a script.

Data Science for Petroleum Engineering - Part 5.2: Finding and filling missing data

22 August 2017

data science, petroleum engineering, R, well data

NOTE. You can find the PDF version of the R markdown notebook in GitHub at this link. The reproducible R markdown notebook (.Rmd) itself is here. Both are full versions of this LinkedIn article. For the time being, LinkedIn publishing does not support markdown which would make sharing scientific and engineering documents much easier. Mistyped data One of the challenges in cleaning up well data is having uniform and standard well names.

Data Science for Petroleum Engineering - Part 5: "Transforming Excel well raw data into datasets."

18 August 2017

data science, petroleum engineering, R

One of the big challenges of this new era of data science. machine learning and artificial intelligence is getting unhooked from the habit of working with spreadsheets. They have been around for 30+ years and were awesome. But spreadsheets - or worksheets - do not scale well with massive amounts of data; or continuous streams of data; or other characteristics that are key for taking good and sound decisions such as reproducibility.

Data Science for Petroleum Engineering - Part 5.1: Data Introspection with R

17 August 2017

data science, petroleum engineering, R

NOTE. You can find the PDF version of the R markdown notebook in GitHub at this link. The reproducible R markdown notebook (.Rmd) itself is here. Both are full versions of this LinkedIn article. For the time being, LinkedIn publishing does not support markdown which would make sharing scientific and engineering documents much easier. Transforming Excel well raw data into datasets This section is about getting familiar with our data.

All Blog posts by date