Thoughts on Open Source

opensource-svg

I used to not understand open source.

Why would people give away their source code for free? Going through the software revolution of the 90s, Microsoft was the king of software. I followed the Microsoft’s decimation of WordPerfect, Borland, Lotus 1-2-3 and Netscape as a kid. And Microsoft didn’t give away software for free, no, let alone source code! Microsoft charged folks for Office, Windows, Visual Studio, etc. To their credit, they still do to a certain extent. So to me, it all seemed counter-intuitive – something that was free, in my mind, would not see any quick progress due to the lack of economic motivation behind it. My overall train of thought could be summarised by this grumpy nerd’s post.

Then Linux Torvalds came along with this :

Hello everybody out there using minix –

I’m doing a (free) operating system (just a hobby, won’t be big and professional like gnu) for 386(486) AT clones. This has been brewing since april, and is starting to get ready. I’d like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things).

— Linus Torvalds

And it’s amazing how others have contributed to the project ever since. Linux is probably the most successful example of open-source software. From the Linux kernel, countless distributions have emerged. Heck, even Android has the Linux kernel in it. Red Hat, CentOS, Ubuntu, Debian, Linux Mint they all share the linux kernel, which is actively being maintained by the Linux Foundation.

There’s a lot of open source software out there, and it’s all a beautiful mess. Companies leveraging on open-source software have been successfully built. Red Hat is a prime example of this, which generates revenue through support and training contracts. http://www.zdnet.com/article/red-hat-nearing-1-billion-in-revenue-not-bad-for-free-software/ And did you know that WordPress, the blog software that you are reading this on, is open-source?

The problem with proprietary software is this : When you don’t know what’s going on, you don’t know what’s going on. Proprietary APIs usually hide the implementation details of a function, so you are at the mercy of whoever holds the proprietary software implementation. You would be in danger even. I’m sure Snowden would agree. With open source, all the implementation details are out there, whether you want to understand it or not. Furthermore, with proprietary software, once the company decides that it is no longer in their financial interests to continue with the work, it is thrown away.

I’ve concluded that to make large scale software without a whole Fortune 500 company like Microsoft or Google behind you, open source is the way to go. I use ZeroMQ in my work, and this is a great example of this. The author has successfully built a community from which many other projects have sprung.  Once folks find that the open source code is useful to them, they will have it in their hearts to contribute back. Even if it is just fixing a bug or two.

My other open source loves are CSound and  AudioKit , all great audio programming open source projects from which one can learn. AudioKit is fairly new, but has enjoyed quick growth due to many developers wanting a Swift audio development framework (pun intended).  It was the CSound code that inspired me to delve into DSP code.

Revenue for the little guy? I’d like to think of somehow who buys stuff I make as paying me to keep it going. It is not easy maintaining a piece of software amidst the ever-changing tech landscape.I would say it takes more energy to maintain software than to shit one out.

Open source is simply a developmental methodology that was ahead of its time. It’s a methodology that places community over software. It’s a reflection of the human need to contribute and make a difference. I that software, like nature, given enough time undergoes evolution. Great software stays and gets better with time.

Valgrind – Suppressing CUDA/ZMQ/IPP/OpenMP Errors

Valgrind is great, but it doesn’t recognize some calls like CUDA’s. This means that Valgrind frequently reports these as leaks, even when they are legitimate calls. Even a simple cudaFree() can cause Valgrind to complain.

You call the suppression file like this:

> valgrind --suppressions=ippcuda.supp ./yourProgram

What is does is to basically ignore these errors that valgrind complains about. The downside is that you may miss some legitimate leaks in CUDA, for example. For that, you can use cuda-memcheck , but that is really, really slow. Boo NVIDIA.

##----------------------------------------------------------------------##
# ZMQ Suppresions

{
<socketcall_sendto>
Memcheck:Param
socketcall.sendto(msg)
fun:send
...
}
{
<socketcall_sendto>
Memcheck:Param
socketcall.send(msg)
fun:send
...
}

##----------------------------------------------------------------------##
# Intel Suppresions

{
 <insert_a_suppression_name_here>
 Memcheck:Cond
 fun:__intel_sse2_strrchr
 fun:_init
 fun:_dl_init
 obj:/usr/lib64/ld-2.17.so
 obj:*
 obj:*
 obj:*
}

{
 <insert_a_suppression_name_here>
 Memcheck:Cond
 fun:__intel_sse2_strrchr
 fun:DynReload
 fun:ippSetCpuFeatures
 fun:_init
 fun:_dl_init
 obj:/usr/lib64/ld-2.17.so
 obj:*
 obj:*
 obj:*
}

{
 <insert_a_suppression_name_here>
 Memcheck:Cond
 fun:__intel_sse2_strrchr
 fun:DynReload
 fun:ippSetCpuFeatures
 fun:main
}


{
 <insert_a_suppression_name_here>
 Memcheck:Cond
 fun:__intel_sse2_strrchr
 fun:_init
 fun:_dl_init
 obj:/usr/lib64/ld-2.17.so
}

{
 <insert_a_suppression_name_here>
 Memcheck:Cond
 fun:__intel_sse2_strrchr
 fun:DynReload
 fun:ippSetCpuFeatures
 fun:_init
 fun:_dl_init
 obj:/usr/lib64/ld-2.17.so
}

{
 <insert_a_suppression_name_here>
 Memcheck:Cond
 fun:__intel_sse2_strrchr
 fun:DynReload
 fun:ippInit
 fun:main
}

##----------------------------------------------------------------------##
# OMP Suppresions

{
 <insert_a_suppression_name_here>
 Memcheck:Leak
 match-leak-kinds: possible
 fun:calloc
 fun:_dl_allocate_tls
 fun:pthread_create@@GLIBC_2.2.5
 obj:/usr/lib64/libgomp.so.1.0.0
 fun:_ZN9AutoFocus8startPGAEP7Ipp32fcfiiPf
 fun:main
}

##----------------------------------------------------------------------##
# CUDA Suppresions

{
 <alloc_libcuda>
 Memcheck:Leak
 match-leak-kinds: reachable,possible
 fun:*alloc
 ...
 obj:*libcuda.so*
 ...
}

{
 <alloc_libcuda>
 Memcheck:Leak
 match-leak-kinds: reachable,possible
 fun:*alloc
 ...
 obj:*libcufft.so*
 ...
}

{
 <alloc_libcudart>
 Memcheck:Leak
 match-leak-kinds: reachable,possible
 fun:*alloc
 ...
 obj:*libcudart.so*
 ...
}

If this doesn’t suit you, you can print your own suppressions using.

./valgrind --gen-suppressions=yes ./yourprogram

Valgrind will then generate suppressions catering to the particular error you have.
Don’t go around suppressing legit valgrind leak detections though!

Here’s my own suppression file which includes the above, hope it helps.
https://onedrive.live.com/redir?resid=692F268A60881F2D!22968&authkey=!ANsb8IMA9e8lkOw&ithint=file%2csupp

Useful ZMQ Message Helper Classes

If you use ZMQ in C/C++, be sure to check this out.

https://github.com/pijyoi/msg_parts

Why use this, u say?The advantage is that msg_single_t/msg_multi_t objects on the stack get cleaned up automatically. It helps to clean up a lot of boilerplate code.

Previously,

zmq_msg_t msg; 
zmq_msg_init(&msg); 
zmq_msg_init_size(&msg, sizeof(int)); 
int payload = zmq_msg_recv(&msg, zpipe, 0); 
zmq_msg_close(&msg); 

Can now be replaced by

msg_single_t msg(sizeof(int)); 
int payload = msg.recv(zpipe, 0); 

Closing the msg is done for you when msg goes out of scope. This helps to reduce memory leaks / zmq errors due to not closing the message properly. Pretty, pretty code!

ZMQ – Invalid free() / delete / delete[] / realloc() / __libc_freeres error in Valgrind

Below is some simple code with ZMQ.

zmq:context_t context(1);
zmq::socket_t MPC1Socket(context, ZMQ_DEALER);
MPC1Socket.setsockopt(ZMQ_IDENTITY,”test”, 5);
MPC1Socket.connect(“tcp://localhost:8001);
If (MPC1Socket != NULL) MPC1Socket.close();
If (context != NULL) context.close();

Strangely enough, I got the following error when profiling it in valgrind.

# valgrind --leak-check=full --suppressions=./valgrind.supp ./MasterInterface

 

==11063== Invalid free() / delete / delete[] / realloc()
==11063==    at 0x4C2AD17: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==11063==    by 0x5C25A8B: __libc_freeres (in /usr/lib64/libc-2.17.so)
==11063==    by 0x4A246B4: _vgnU_freeres (in /usr/lib64/valgrind/vgpreload_core-amd64-linux.so)
==11063==    by 0x5AFAE0A: __run_exit_handlers (in /usr/lib64/libc-2.17.so)
==11063==    by 0x5AFAE94: exit (in /usr/lib64/libc-2.17.so)
==11063==    by 0x5AE3AFB: (below main) (in /usr/lib64/libc-2.17.so)
==11063==  Address 0x5e7c3d0 is 0 bytes inside data symbol "noai6ai_cached"

 

Solution?

Change localhost to 127.0.0.1. (don’t ask me why)

MPC1Socket.connect(“tcp://localhost:8001);

to

MPC1Socket.connect(“tcp://127.0.0.1:8001);

Not saying that this is definitely the issue when you encounter the freeres problem. But you could give it a try.

ZMQ : Hanging at Context Termination

If you use ZMQ, and am wondering why you are experiencing hanging when when you exit the program, it’s because the default LINGER value for your socket is set at infinity. When closing the context, the program hangs at the context termination routines.

You need to do this.

// Configure socket to not wait at close time
int linger = 0;
yoursocket->setsockopt (ZMQ_LINGER, &linger, sizeof (linger));

Otherwise, you might run into cases where the context termination routine is waiting for the socket to close. The socket will be blocking still at a function say, socket.recv() and will be unable to unblock unless ZMQ_LINGER is set to 0.

NOTE: Bindings such as CZMQ may have already done this for you.