Brick by Brick / Line by Line

python-tricks-that-you-cant-live-without-11-728

As the CEO of a software startup that will be launching in the new year it is frustrating talking to interested parties about what we are doing, as the first question they ask is can I see how it works.

Unfortunately this is currently not possible as we don’t yet have a beta version for people to play with. However, the one thing that keeps my spirit soaring is watching each line of new code added to our GitHub repository.

Just like you build a beautiful building one brick at a time, so do you build beautiful software, one line of code at a time.

Intuition and Synchronicity show their Power

In January this year we presented our seed funders with a dollar and time budget for our new startup PsyQuation (we are unfortunately still in stealth mode so our site has no information (2 more months)).

When we put our budget together we had no idea on the details of the project. Yesterday I spent a lot of time going through the budgets, and I am very proud to say that we are well on target to come in under budget $-wise (despite a 20% AUD devaluation) and on time, time-wise.

How did this happen given all the distractions and crises that have happened since January?

Intuition is something Jung writes a lot about, and I unfortunately don’t have the time to unpack it from a theoretical point of view. The key point is that when you are an experienced professional you are able to do amazingly complex calculations unconsciously which expresses itself as intuition. Take a grandmaster chess player, it takes super computers with enormous processing power and speed to think like a grandmaster. While the grandmaster can do the calculations with no effort and in seconds drawing on their experience and their brains ability to process deductive reasoning.

The synchronistic event I wish to discuss is the reference I made on these pages a few months ago regarding my desire to learn the Python computer programming language. Why did this interest me? I had already embarked on learning R the more in vogue data science language, yet something in my psyche kept pushing me to learn Python.

Well what do you know, through no influence of mine, the language we have chosen to be the universal language across our companies software development is Python. That is not to say we won’t write things in C++, or other database languages at times, but essentially the glue with all the code will be Python. As the CEO of the company it is immensely powerful that I am familiar and can write some code in the language of the company I represent. A few months ago I often wondered to myself, “why”.

I now know Why!

Python Progress

A quick update on my Python coding progress.

I am proud to say that I am actually writing some code that is actually doing stuff in Python. Boy starting something new is time consuming, doing the most mundane tasks that one could do in Excel in a flash takes absolute hours to do in a new language.

I also constantly feel why I am doing this when I know how to code many of these functions in R.

After a couple of weeks of learning I have now accumulated some knowledge which has been further enhanced with the arrival of a few more Python textbooks last week. In summary I am fast falling in love with the Pandas data library in Python which is like learning a whole new language on its own. Pandas has one of the most comprehensive data manipulation functions a data scientist can dream of. I love it, and I am sure in the weeks and months to come if I stay with the learning I will be doing really cool things with the data I come across.

So far I haven’t come across a library like “Performance Analytics” in R, once I come into contact with this type of library in Python then I will feel complete. I believe there is a way to pass R functionality via a wrapper into a Python project. This may be something I will research later today, however my first prize is to stay completely Python native, so anyone with insights into a Performance Analytic type library in Python please let me know.

Massive Python Weekend

I thought I would just summarize some of my early Python experiences.

Let me start by saying it hasn’t been as easy as many of the reviews would have you believe. To begin with the current version in production is 3.4.3 which is built on a completely new framework to the version 2 productions. So to begin with you encounter a debate about whether to start learning with version 2 or 3. You encounter this issue with many of the learning tutorials and video’s.

Once you have made your choice about version and download the Python program you are faced with the issue of an IDE or to stick with the raw “Shell” console. This reminded me of my earlier days with R where you also got a simple console, very similar to the DOS command prompt. The next big issue comes with downloading and installing libraries that are not shipped with the standard python installation.

With R there is no longer the issue with the IDE as RStudio is now accepted as the default IDE and installing new libraries is simply done the CRAN distribution hub doing all the heavy lifting with the built-in RStudio installation tools. Python libraries are typically shipped as binaries and need to be compiled into the base python programme. These are all big obstacles to getting going as a novice. Fortunately there is a great piece of software called Anaconda which updates and compiles most of the library’s in the broader ecosystem making it easier to work with.

So far I have chosen PyCharm as my IDE and code editor of choice. I have also found IPython Notes to be an awesome way to share code and explain each step in a markdown text so there are great editing solutions it just takes some time to get a grip on as there is so much choice.

I have to say I have been struggling a little to unlearn some of the R syntax to handle simple data manipulation and plotting. The literature seems to say that R syntax is very non-intuitive compared to much easier more natural Python syntax. As R was my first language I learned I am finding things the complete opposite.  As a quant it seems to be an essential to work with Numpy and Panda libraries as these guys seem to do all the heavy lifting in terms of standard functions for our industry. Because there is so much buy in to these libraries I plan on focusing a lot of my learning around these functions and not bother learning how to do things from the native library that can be doing with the call of a simple function in Pandas.

One of my biggest weaknesses and strengths is by nature I don’t like to ask for instructions, I am the kind of guy who likes to learn by myself on my own terms – excuse the arrogane. I almost never read instruction manuals when purchasing new appliances. I simply dive in like the proverbial bull in a china shop and try and work it out. That is how I learned my way around R and that is kind of how I started with Python.

Then it dawned on me I am not a computer science graduate with a good understanding of high level coding, and the different language environments, etc. To this end I have started to go down the more passive and slower approach of going through video tutorials by experts. This was indeed a good move as it has filled in many blanks and given me a far greater context to the subject and stronger building blocks.

The big find for me was Udemy which is an online “university” where teaching is broken into tiny ~5 minute type modules. I have been peppered with discount offerings and bought some excellent courses for $19 a piece some of which typically go for $279. So while this goes against my nature of diving in head first I have spent many hours over the last week “attending” lectures. In order to keep me sane though every couple of hours, I try and solve a real life problem that I have already mastered in R. This is still the best way to learn as it engraves the steps into ones memory far better than passively sitting and watching a lecture.

In conclusion I am starting to feel like I am getting somewhere but know I have a very long road ahead. As a further point of mention I must add that the reason for me wanting to learn a language like Python is that I know I will never be the sharpest quant in the house as many of the complex math problems are simply too difficult for me to master. However Python is far more than just a quant tool it has excellent Web building and general purpose capability that one cannot do well in R and it is these more general capabilities that I am hoping to add to my toolkit.

PS:

Last night I was running some code in R which was doing quite an involved VaR calculation over 1yr of daily data. The calculation was not running on the time series of the equity curve but on the open positions at each point in time and was taking my computer about ~1hr to run. I am very interested to know how long it would take to run in Python. I suspect it will be quicker, however as Python and R are not compiled languages they will still run much slower than say C++ or other compiled languages. All of this for another day, I would be happy to simply achieve “Hello World” in Python for now  😉