Row major and Column major Explained, using Python against Matlab

In both cases, a 2d array of 2 rows, 4 columns is created.
Then, it is reshaped to 1 row, 8 columns.
This clearly demonstrates how Matlab stores the array data column-wise, and Python stores the array data row-wise after it is flattened out to 1d.

Matlab (Column Major)

Screen Shot 2017-06-04 at 10.51.20 AM

>> a = = [1 2 3 4 ; 5 6 7 8]

a = 
 1 2 3 4
 5 6 7 8

a = a(:)

a =


Python (Row-major)

Screen Shot 2017-06-04 at 10.51.13 AM

>> import numpy as np
>> a = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])
>> a
array([[1, 2, 3, 4], 
        5, 6, 7, 8]])

>> a = a.tostring() # shows the memory arrangement, in a string

Out[] : b'\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00\x04\x00\x00\x00\x05\x00\x00\x00\x06\x00\x00\x00\x07\x00\x00\x00\x08\x00\x00\x00'

If you look at the string printed, it shows that the elements are arranged in the fashion 1, 2, 3, 4, 5, 6, 7, 8.

Why is this important to know, you ask?

For optimization, it is important. In Python, C/C++, the elements will be laid out contiguously row-wise in memory. So you should strive to process the elements row-wise. The entire row could be copied to cache and worked on from there by the OS. If you processed the elements column wise, contrary to how it is laid out in memory, you would incur expensive context switching as the OS copies in the elements you require into cache for processing for every column-wise element you process!


Conquer CSV files in Python


CSV files are common for data manipulation in Python, in cases where you extract the data from an excel sheet. Here is a short tutorial on how to extract some data out of a csv file, along with other nifty tricks along the way.

1) Easy Binary Conversion

You can use this to convert to binary.

your_binary_string = "0100000010001"
int_value = int(your_binary_string, 2)

Of course, you can extend this to octal, etc.

2) File reading and list comprehension

Suppose you have a whole csv of binary numbers. You need to read it out to python as a list.
Read the csv as string, and convert it to int easily.

Your csv data looks like this :


and you just want to get the values in bold. Meaning putting each of the the fourth and third column values in a tuple, converted to integer value.

import matplotlib.pyplot as plt
import numpy as np
import csv

filename = 'yourfile.csv'

with open(filename, 'rt') as csvfile:
 reader = csv.reader(csvfile)
 # the following line uses list comprehension to iterate through every row in the csv
 # and creates row number of tuples in a list
 # You can call next(reader) to skip a row, for example if the 1st row is just labels
 iq = [(int(row[4], 2), int(row[3], 2)) for row in reader]

You get something like this:

[(4079, 63914), (4353, 61824), (3683, 62088), (3813, 61592) .... ]

At this point, if for some reason you want to get all the 1st values in each tuple in one list, and the 2nd values in each tuple in another list, you can do:

i, q = zip(*iq)
i = list(i)
q = list(q)

You will get for i :

[4079, 4353, 3683, 3813, 3231, 3201, 2750, 2550, 2201, 1874, 1559, .... ]

and q:

[63914, 61824, 62088, 61592, 61459, 61228, 60994, 60880, 60655, 60587... ]

3) Writing CSV files

With the previous i and q lists you extracted, you can now write it out as a csv.

with open('complex_out.csv', 'w') as csvfile:
 fieldnames = ['i', 'q']
 writer = csv.DictWriter(csvfile, fieldnames = fieldnames)
 for row_num in range(len(i)):
 writer.writerow({'i':i[row_num], 'q':q[row_num]})

Your csv file looks like this :


3) Converting data types

From the above data, I know that my number is really a 16bit binary representation.
Now, I am told that this is a 2’s complement representation.
So each tuple should be really (int16, int16), with +ve and -ve values possible. Fortunately Python allows us to do this easily.

iq = np.array(iq, np.int16) # creates an array with iq existing values, converted to int16
array([[ 4079, -1622],
 [ 4353, -3712],
 [ 3683, -3448],
 [ -567, 5301],
 [ -216, 5329],
 [ 146, 5332]], dtype=int16)

Now we want to convert it to complex64, for further processing down the line.

sig = iq.astype(np.float32).view(np.complex64) # convert the values to float32, before viewing it as a complex64 (2 float32s)
array([[ 4079.-1622.j],
 [ 4353.-3712.j],
 [ 3683.-3448.j],
 [ -567.+5301.j],
 [ -216.+5329.j],
 [ 146.+5332.j]], dtype=complex64)
sig = sig.ravel() # flatten it to 1D
array([ 4079.-1622.j, 4353.-3712.j, 3683.-3448.j, ..., -567.+5301.j,
 -216.+5329.j, 146.+5332.j], dtype=complex64)

Hooray, now we’re ready to do further processing on this data!

Python: Using ioloop To Send to 2 Protocols Without Threading

ioloop is very useful for pre-scheduling functions to be executed in the future. In this manner, you can simulate interspersing sending to 2 protocols without threading. (in this case, “serial” and “tcp” at 100Hz and 1000Hz respectively.)

In this simple code, io_loop.add_timeout keeps pre-scheduling events to happen in the future. Do note that this all runs in one thread, so make sure your sending functions printTCP() and printSerial() are never blocking. I use this for simple use cases where I am just reading from a file and sending to two different sources. Also, in my code I use a lot of zmq, and only required the simple use cases of zmq.eventloop. For more advanced use cases, you may want to take a look at tornado.ioloop. (on which pyzmq’s event loop is based on)

import functools
import datetime
from zmq.eventloop import ioloop

def printTCP():
 print('Sent TCP')

def printSerial():
 print('Sent Serial')

io_loop = ioloop.IOLoop.instance()

# Send serial messages at 100Hz
serial_deadline = datetime.timedelta(seconds=0.1)
for x in range(1000):
 io_loop.add_timeout(serial_deadline, functools.partial(printSerial))
 serial_deadline += datetime.timedelta(seconds=0.1)

# Send tcp messages at 1000Hz
tcp_deadline = datetime.timedelta(seconds=0.01) # separate deadline for sending ins pulses
for x in range(10000):
 io_loop.add_timeout(tcp_deadline, functools.partial(printTCP))
 tcp_deadline += datetime.timedelta(seconds=0.01)


You will get something like this:

 Sent Serial
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent Serial
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent Serial
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent Serial
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent Serial
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent Serial
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP
 Sent TCP

Super Easy Sequence Diagrams – PlantUML is a very easy to use sequence diagram maker that is free. It has a extremely intuitive syntax that will allow you to create complex diagrams within minutes.

There’s a online version here :

Take for example :


participant FDC_FPGA

box "FPGA" #LightYellow
 participant FDC_FPGA
end box

box "MPC" #LightBlue
 participant Kernel
 participant User_Space
end box

User_Space -> Kernel : Wait Data Ready ioctl
activate User_Space
activate Kernel
FDC_FPGA -> Kernel : Data Ready Interrupt
Kernel -> User_Space : Data Ready Awake
deactivate Kernel
deactivate User_Space
User_Space -> Kernel : Setup Read Descriptors ioctl
Kernel -> FDC_FPGA : Set Registers
User_Space -> Kernel : Enable DMA Start ioctl
Kernel -> FDC_FPGA : Set Registers
User_Space->Kernel : Wait DMA Done ioctl
activate Kernel
activate User_Space
FDC_FPGA -> Kernel : Dma Done Interrupt
Kernel -> User_Space : Dma Done Awake
deactivate User_Space
deactivate Kernel


This gives a very nice diagram that is ready for presentation :


I use it all over the place for Software Design Documents. The best thing is – it has “source code” , so anytime there’s a change done, all I have to do is to use the “source” to regenerate the diagram. Happiness!

NCD: 2016 Macbook Pro 15″ Review! (refurb)


Got me a 2016 MacBook Pro 15″. Got it refurbished but it is really looks good as new. I really couldn’t justify paying the full price for what you get. No complains here, Apple’s refurbished stuff is top-notch.

Here are some of my thoughts on it after the one month honeymoon period.


Apple knows how to make desirable, beautiful hardware. And the new MacBook Pro is no exception. Apple stuck to the brushed aluminium feel and the whole package looks elegant. Not much to pick at regarding looks.


The new low-travel keys take some getting used to – I’m actually still not used to them. I like chiclet keys, but these are even lower travel than those. It almost feels like they are trying to steer you in the direction of on-screen keys on a digital screen by introducing these. Perhaps such keys are less failure-prone?

They do work functionally, but give off a clickety-clack sound when you are typing. Those with mechanical keyboards may like this “feature” that it has. Overall, I still prefer the feel of the keys of previous models.


The 15″‘s touchpad, or trackpad as Apple would like you to call it, is slightly too big in my opinion. It has built-in palm rejection to prevent accidental triggering of the cursor when you are typing, although I still do mange to trigger it, once in a while. No major complains here, and Apple’s gesture integration with OSx is top-notch, as always.


And here comes the latest gimmick from Apple’s stable. The touch bar. I have to admit, when I just got this machine, I wanted to use the touch bar so much that I was on the look out for stuff appearing on it. But as time went by, I realised that I never actually look at the keyboard when I am typing, let alone look at the touch-bar. In this respect, the touch bar doesn’t serve its intended purpose. It isn’t a selling point for me. If I could get the same model for 200USD less without the touch-bar, I would.

Dongles, Dongles, Dongles!

USB3.1 is excellent technology. You can interface lots of things to it – HDMI / power / just about anything you can think of.  It has a transfer speed of up to 5GBits/s, which translates to 625MB/s. This is comparable to a SSD’s write speed. What this means is that you can expect to write to an external USB3.1 SSD drive with the same speed as you would to an onboard SSD, with both drives being the same.

Apple loves USB3.1, and they loved it so much they put nothing but USB3.1 connectors on the laptop. Even the power adaptor is USB-3.1, which is a shame – I loved the MagSafe laptop power connector on older models.  On the flip side, it also means that you don’t have to lug a power adaptor around, any wall power with usb3.1 socket and be used to charge your Macbook.

The problem is : These ports are ahead of their time. There really isn’t a lot of USB3.1 devices out there at the moment. So for now, we will have to use dongles to convert these connectors to the intended connector type. In time to come, I don’t see why devices not all converge to USB3.1 as a standard connector. It just makes everything easier – but currently it is a case of the real world needing to catch up the the tech.

Last Words

The real reason I got Apple’s laptops is that I make iOS apps – there isn’t a real alternative  that doesn’t involve tons of hair pulling when all I want to do is to be productive.  Regarding this, I got what I wanted. No more waiting for Xcode to load interface builder, or web-page freezing up when I too much is going on in a web-page. Rendering stuff is much faster than on my late 2011 MacBook Pro.

I got six good years out of my 2011 MacBook Pro, and it is the only computer that I can profess to use the term “love” on. In my opinion, OSx is a better OS than Windows. It feels as stable as a Linux-based system, with the software that I need to do my stuff. It is an added bonus that you can plug in a USB keyboard and have it play music software with little latency immediately. The Microsoft lovers will say, but windows runs on all PCs, so it inevitably has to do more heavy lifting compared to OSx. I don’t care. I just want to get things done, and the machine that allows me to do that is my weapon of choice.