Every month, StackOverflow, the brilliant and completely open question and answer site for programmers, releases a Creative-Commons licensed dump of their database. You can download this huge dump yourself, but it’s typically a very large file, and then comes the hassle of getting the XML files into a database of your choice. Now, there is a way to get around those inconveniences, by simply bypassing the whole download procedure and querying the data dump in the cloud! That’s what StatOverflow is for. Just go to the sandbox, enter an SQL query, and voila! Also, there’s a quite helpful meta.stackoverflow post which explains the anatomy of the dump, so that querying it is much easier. Enjoy!
Tag Archive: Programming
Numerous friends have asked me about this whole Twitpocalypse thing, as they just don’t get it. “What’s this whole deal with signed integers?”, they ask. Well I’m here to clear that up, as I do programming, and in programming, if you don’t know such basics of computer science, you’re doomed.
What the predicters of the Twitpocalypse were referring to when they mentioned the limit of 2,147,483,647, is the limits of the signed integer data type. The signed integer is a 32-bit data type, so it can hold only so large (or so small) a number – in this case, the highest number the data type can preserve in 32-bits is 2,147,483,647, and the lowest number is -2,147,483,647, respectively. This is what causes the Twitpocalypse problem, as well as the foreseen Unix Year2038 problem (where Unix epoch time will cross the limit, and distros using the 32-bit data type will fail.)
Each tweet in the Twitter infrastructure has its own numeric ID. Thus, programs that invoke the Twitter API’s ID interface into a 32-bit variable fail, as the number of tweets has passed the limit. When this happened, some applications failed, others continued working, depending on their implementation of this feature. Of course, updates were quickly rushed to the failing products, so that users wouldn’t be discouraged or affected for too long. Most applications now use the 64-bit integer datatype (commonly referred to as long or int64) for the ID variable, which supports numbers from −9,223,372,036,854,775,808 to +9,223,372,036,854,775,807 – in other words, pretty large. Until then, we’re safe, but if we manage to cross that 9 quintillion mark (well, it’s certainly possible! 
), we may be forced to convert to the 128-bit integer time, or even storing numbers as strings (the datatype for any types of characters, mostly letters), but I think computing will have changed a bunch by then, so that we might not even think of such a problem. That is, if Twitter lasts till then, or maybe the world will have been sucked into a black hole. We’ll see.
(On a side note, last year, I created a VeryLargeInt class for storing huge numbers (100 digits long can be stored with this, or even more!) by using the string data-type, and I also built algorithms for computation with strings, including most common mathematical operators. I may publish this project here in the future.)
If you’re a .NET developer, admit it: you’ve at least once felt curious about how the underlying APIs behind the .NET Framework are coded, how certain functions are implemented, or whether one core function is more efficient that another. Well, this is where the trusty program named .NET Reflector comes in. As the title suggests, Reflector uses .NET reflection and ILDASM processing. It is the decompiler of the .NET world, and allows you to view source code of DLLs or executables in multiple languages (including C#, VB, C++, Delphi, IL, and more) with ease. It is very interesting to examine how the brilliant programmers at Microsoft sculpted the inner workings of the programming framework that drives many applications today. With this tool, you can accomplish just that and much much more. The Reflector was originally created by Lutz Roeder, but has been acquired by Red Gate Software. The program is free for all to download and use, and it can even Reflect upon itself! Now that is true irony. You can snag a copy over at the Red Gate site.