Category: Tech & CS

Mongo Native TTL

Turns out MongoDB supports a native TTL mechanism, natively removing documents at or after a certain amount of time, similarly to Redis’s auto-expiry. Here’s an example usage flow:

>{time: new Date(), name: "Jones"})
{ "_id" : ObjectId("53ce96f03fdf121aa2e36622"), "time" : ISODate("2014-07-22T16:53:04.001Z"), "name" : "Jones" }
> { "time": 1 }, { expireAfterSeconds: 5 } )
{ "_id" : ObjectId("53ce96f03fdf121aa2e36622"), "time" : ISODate("2014-07-22T16:53:04.001Z"), "name" : "Jones" } //still there, less than 5 seconds have passed
> //wait 5 seconds
> //empty

Mongo Awesomeness FTW. 

Mongo Console BI syntax essentials

When messing around with the Mongo console I’ve found that Mongo is not just JavaScript, and the JS-foo and standard APIs you might be used to in the browser or in Node are not always available – where’s console.log, for Console’s sake?

Here’s a head start on how to perform some basic manipulations, assuming prior familiarity with JavaScript and Mongo themselves.

So suppose:

  1. You’ve got your data spread around mutiple collections (supposed they’re named ’employees1′, ’employees1′, ’employees1′),
  2. You want to query all the fields of a nested document (say, the field ‘city’ of nested document ‘address’, which not all the users have).

You might want something like this:

mongo > res = []; 

mongo > for(i=0;i<4;i++){ res=res.concat(db.getCollection('coll'+i).find({"address": {$exists: true}}).toArray()); }

mongo > res.length //number of employees that have an 'address' sub-document.

mongo > res.forEach(function(x){print(}) //print out all the cities

Hooray for science.

Node Background Processing Awesomeness

Unless I’m misunderstanding something, a Node.js callback can replace a full system architecture of handling asynch background processing.

Suppose you have a webapp route that initiates a heavy background task: POST ‘/do_heavy_lifting’

You don’t want your users to wait forever while you process; that’s sucky UX and blocks your app’s precious resources. So, you might drop an event down to some messaging queue (say, Rabbit), and write up a worker to listen to that queue and then perform the heavy lifting in the background. Now you’ve got to maintain the worker, the queue, the event, and synchronize all the different parts together… This is only an issue once you reach a certain scale, but then — it’s an issue, once you reach a certain scale.

However, there is a better way.

As you surely know, you can use Node.js’s callback awesomeness to perform heavy background processing (codename “I/O”) without blocking the code execution flow. However, if you respond to the request without waiting for the I/O to finish, you can even respond to the user – thus freeing the connection – while allowing the background processing to finish in its own time!

This allows you to handle arbitrarily heavy background tasks while immediately freeing the response, both clearing the HTTP connection and responding immediately to the client. Holy smokes!

Sample code, using writing content to a file as an example of heavy lifting (observe the response.end outside the callback):

// Require what we need
var http = require('http');
var fs = require('fs');

function now(){
    var d = new Date(); 
    return d.getHours()+d.getMinutes()+':'+d.getSeconds()+'.'+d.getMilliseconds();

var app = http.createServer(function(request, response) {
  response.writeHead(200, { 'Content-Type': 'text/plain'});

  fs.writeFile('baz', 'baz contents: '+now(), function(err) {
    console.log('created file. Now running arbitary background code');            
    console.log('before Second Reponse:'+now());
    response.end('Redundant response: '+now()+'\n'); 

  console.log('before First Response :'+now());
  response.end('First Response : '+now()+'\n');

app.listen(1337, 'localhost');
console.log('Server running at http://localhost:1337/');

Copy the above into foo.js, $ node foo.js, and (separately) $ curl localhost:1337. Notice:

  1. You will always get ‘First Response’, which is expected.
  2. The ‘redundant response’ is never used. In fact we can comment it out; same behavior. (It’s redundant in our case because it’s a background operation, and we don’t need it to finish in order to respond to the user.)
  3. The I/O operation does get executed, as well as its callback (which can be arbitrarily extended).
  4. Under ‘arbitrary background code’, we can now execute whatever we want — the client has already receiving its response.

You have just implemented a full-blown background asynch processing paradigm, with about a line and a half of code and some Node.js-Fu.

Copy Local Files into Remote Server through n>1 SSH Hops

Copy your local files into a remote environment with an arbitrary number of hops using the following:

$ rsync -have "ssh -A -t user@middleman ssh -A -t user@destination" source_path :target_path

As in, copy your local directory ‘foo’ to the root path of your staging server using:

$ rsync -have "ssh -A -t user@middleman ssh -A -t user@destination" foo :


  1. If you are copying a directory, remember the target path should be the target directory’s parent folder.
  2. If you are using this (‘hot-deploying’) instead of a traditional ‘deploy’ from a version-controlled repository, exercise the appropriate caution: verify your files have been copied to the correct location, remember they haven’t necessarily been source-controlled, and so on.
  3. I learned this from:

Hooray for Science!

Ruby source_location

I was having a very hard time finding how/where a specific legacy method was defined in our Ruby codebase. Every conventional method failed to find it, including searching the entire project for the method name.

To my rescue came an in-built method of locating the source definition on a method in run-time:


Available in Ruby 1.9+.

(Indeed, the method name is dynamically defined by a grep on /in_cents/ columns using dark Ruby such as ​class_eval. Whodathunk.​)

My Git Book & Sandbox

tl;dr: I am not satisfied with the Git tutorials I have seen so I have written one myself. You can see its draft here:

We have had 14(!) new developers join our team(s) in the last 8 weeks. Every single one of them will be using Git from day one. We have an aggressive deployment process using a centralized workflow – no dictator/lieutenants or any of that, every single developer pushes and pulls to and from the main repository. Our new devs are new – often new to Unix, new to Web development, and certainly new to Git.

Git is hard. Git is complicated. All the more so as the team size grows. Every single dev is expected to handle the complications of branching, staging, diffing, committing, merging, remotes, resets, rebasing, conflict resolution, and so on. HEAD. Index. Stash.

Git follows the Unix philosophy – everything is possible, but nothing is obvious. Git’s “porcelain” or ‘surface’ commands would be considered ‘plumbing’ anywhere else. One solution would be to use a GUI, but that would just be hiding the problem (though perhaps this option should not be dismissed so easily).

Regrettably, it has been my experience that most Git documentation and tutorials are either of the ‘Hello Branch’ variety or of the ‘git plumbing internals’ persuasion, and fail to hit the balance of robustness and accessibility new devs need. This is par for the course with many complicated/technical topics: those who understand it well, struggle to explain it to those less comfortable. You can always tell people to read the man pages, but when it’s your devs and your code on the online, fixing the problem is more important displaying your hard-coreness. Otherwise, new devs just pick up the minimum they can/need in order to survive, and (ab)use Git to their own (and our) detriment.

Lastly, explaining something is always the best way to learn, and explaining Git has been a great for me to improve my Git understanding.

So anyway, I am undertaking a humble effort to create a better Git tutorial. This will include:

  • A single-command setup for a local Git sandbox, so devs can experiment with advanced command without fear of destroying their (or others’) work.
  • A step-by-step walk-through of important Git commands and settings, explaining the reasoning and practicing together in said tutorial.
  • An (admittedly) opinionated dictation of the correct workflow when working on a feature.
  • A (hopefully) reference-style index for those just coming for a quick ‘how do I do that’.

Necessarily, many of the issues will be simplified or approximated, to facilitate understanding over exactness.

I have personally found myself asking about and later explaining these topics over and over. It took me some time to come around to creating my own Git sandbox to practice. Hopefully this will at least help some people, some of the time.

Naturally, this is not the first or last Git tutorial, sandbox, or even book. That’s fine – I am not trying to compete with anyone else, just fill the void my personal way, which I felt has been lacking.

Further drafts will hopefully be prettified and strengthened. Contributions will of course be welcome.

Optimal Notes Strategy For Productivity Freaks

I’ve got lots of stuff to remember so I use notes and I needed a system that was better than any existing tool that I knew of, so I spent a some time thinking about it and now I have a system and it is BOSS.

Use-cases are:

  1. Find a note by name (like ‘git’ or ‘deploy_process’ or ‘electricity bill’)
  2. Find a note by content
  3. Edit, save, sync note to cloud
  4. Access anywhere, anytime, any device
  5. Port all my notes somewhere else in 2 years

The requirements (must be…):

  1. Super-fast to find a note by searching for name.
  2. Super-fast to find a note content.
  3. Super-fast and easy to edit a note and sync to cloud, all without using mouse.
  4. Available online via web, all platforms
  5. Easily exportable (No lock-in)
  6. Complete revision history

Failed attempts: Evernote and GoogleDocs. Both are too heavy, cumbersome, and slow. GoogleDocs is bottlenecked by the network and is in-browser, which is crowded enough. Evernote has a lock-in, no revision history, and is basically über cumbersome. They might work for you if you’re infinitely patient but I am kind of the opposite.

Best solution:

  1. Open a folder in your Dropbox folder. Call it ‘notes’.
  2. Use your favorite text editor (which should be Sublime Text) to find, add, and edit simple text notes.

Shazam, you’re done. Let’s examine how this answers the requirements and use-cases.

  1. Find a note by name: ST’s fuzzy-search for file names is famously boss.
  2. Find a note by content: Ditto, ST’s search-function is famously sweet.
  3. Editing notes: By definition, it’s already your favorite text editor. Saving syncs it automatically to the cloud (dropbox).
  4. Access anywhere, anyhow:, and you’re golden.
  5. Exportable: it’s just a folder of text files! Evernote, Dropbox, and Google[Docs] may come and go (not to mention other app solutions like SimpleNote, etc.), but text files will be here for a long time. Whenever you want you can copy the whole folder off to your folder, or your SugarSync magic briefcase, and edit the files with vim instead of Sublime. No lock-in, ever.
  6. Dropbox saves all your revision history.

As you can see, these cover the requirements perfectly. To me, this clearly outclasses using GoogleDocs or Evernote for any personal use. I’ve been riding this for about 4 months at time of writing, and it’s been a dream. Anything I need to remember can easily be jotted down to the relevant note and retrieved at the speed of light (well, at the speed of Sublime Text, but it’s pretty damn close). Zero upkeep. Totally free. And I know that when I grow tired of Sublime (and/or switch computer, OS, polar meltdown, etc.) my notes will be there for me. This very post’s draft was written in my ‘blog’ note, where I keep a list of topics and first drafts of posts.

Basically this is has just been riding on the awesomeness of Sublime Text and Dropbox. You lose tiny things like clickable links and rich-text formatting, but those are for girls anyway.

Extra points for using ST+Dropbox for notes::

  • Use a specific Sublime theme for your ‘notes’ folder, so you can easily tell the ‘notes’ Sublime instance from code instances. I use Solarized (Light) because I like the light/dark contrast with code’s Monokai.
  • Set a syntax highlighting that complements free-text notes. I use AppleScript for no reason at all.
  • Organize your files into folders, if you feel like it, but with Sublime’s great search capabilities – it doesn’t really matter. I’ve got 150+ notes by now on everything from Apple to ZenDesk and don’t need the folders.

So: never remember anything again. You’ve got your computer to remember, now that you’ve got your llama-whooping notes mechanism set to retrieve and update notes at whiplash speed.

Now, go forth and multiply. Or at least use efficient notes.

Git Rebase for Dummies

In the following post I will go over how and why to use git rebase instead of git merge, even if you don’t completely understand how and what rebasing does.

We will try to explain in layman’s terms (e.g., without relying on the term ‘HEAD’) what’s happening, and most importantly, why you should drop your fear of rebase and learn to love it. If your Git-fu is strong and you’re a Git-ninja, cherry-picking all over the place, you won’t need this. However, if you’re a smart developer but learning Git has been a trip down the rabbit hole, read on.

Specifically we will cover the most common use-case, where you are working (alone) on a feature branch and have finished your feature, and you wish to now push your code to master. Perhaps you might ‘merge’ from master (as many naive guides suggest), but you’ve heard people say you should be rebasing. How? What? Why?

We will cover the following:

  1. Base workflow when rebasing in git
  2. In-depth explanation of what rebase does, in easy terms
  3. Conflicts – and why you shouldn’t fear them
  4. Summary

1. Git rebase workflow cheatsheet:

  • git checkout master
  • git pull
  • git checkout branch
  • git rebase master (if you have conflicts, see below.)
  • git log (see your branch commits on top)
  • git diff master
  • git push –force (unless someone else has pulled this branch. This will mess them up.)

(if ready to deploy:)

  • git checkout master
  • git merge branch
  • git push

2. git rebase, step by step:

  • git checkout master
  • git pull

As you may have heard before, *pull* is actually *fetch+merge*. Anyway, in our case master is now up-to-date.

  • git checkout branch
  • git rebase master

The crucial step, of course. What rebase does is it puts the branch commits on top of master’s commits., instead of merging them together.

Let’s explore the differences between merge and rebase. Suppose master has commits A-B-C-D, and your branch includes your additions, commits E-F. After merging, we will have all the commits together. Hooray? No, wait – the order in which the commits appear will depend on at which point we branched off. Our commit log might look something like A-B-E-F-C-D, despite the last commits de-facto being E-F. The end result (the code) will be the same, but the commit log is deceiving. (In fact, in the case of merging there will also be a redundant ‘merge commit’, but let’s ignore that for now.)

When merging, if we had branched off from A, the end result after merging would be A-E-F-B-C-D. If we had branched off of B, if would be A-B-E-F-C-D. If we had branched off of C, it would be A-B-C-E-F-D. If we had branched off of D, it would be A-B-C-D-E-F. And this is what we want, of course — after ‘adding’ our branch to master (and pushing), we would want the commit log to reflect our changes as being added ‘on top’ of everything else, since logically that’s what happened for master.

Enter rebase. What rebase does is ignore the point at which you branched off. It grabs the current ‘last commit’ of master, and puts all of your commits on top of it. So in effect, it simulates as if you had just now branched off master, done all of your commits, and then merged back. In the above scenario, it will ensure our commit log will be A-B-C-D-E-F. Awesomeness!

Bottom line is if you merge, the post-merge commit log would be something like A-B-E-F-C-D, where B is the commit where you branched off, E-F are your commits, and C-D are any commits pushed to master between the time you branched off and the time you merged.

But if you rebase, the post-merge commit log will *in any case* be A-B-C-D-E-F, where *your commits* E-F are *on top*. Why is this important? For you as a developer, it usually isn’t. I did my commits, merged & deployed, thank the Lord. But think of your coworker: He sees you’ve pushed code to master and goes to your github repo to view the commit log to see what’s changed. Bah humbug! If you’ve ‘merged’ the top commits he sees are C-D, which aren’t the new commits – they were already there. How can he even know what’s changed? It’s very difficult, up to impossible. However, if you rebase, a quick look at the git log will show the newest commits on top. Excellent and shiny.

Now we can know exactly what’s changed, revert if necessary, etc. This is also relevant in the case of Pull Requests, and any other situation in which you inspect the commit log. To review, you should now be able to understand the excellent documentation and visualization at and

  • git log Yes! Our commits are on top. We can easily see exactly what we’ve changed, by commits. To stress the point, when you ‘merge’, the top commits in the commit log will be the last commits pushed to master by *anyone*, not necessarily yours – very confusing.
  • git push –force (unless someone else has pulled this branch. This will mess them up.)

If we rebase over master and then push to our branch, we might have to override a previous push to the branch, since we’ve rewritten history (moving our commits to the ‘top’). The easiest way to overcome this is by –force’ing our code into the repo. Obviously this might create issues for anyone sharing our branch, which is why this method is advocated only if you are not sharing this branch with others.

3) Conflicts

You might have conflicts during the rebase. If this happens, stay calm and follow git’s instructions. Git is here to help you (and if we freak out and lose our marbles, we can always git rebase –abort and go back to square 1.)

As always, Git is a little (too?) verbose. But it helpfully CAPSLOCKS the important part: And at the end, it will (vaguely) what you need to do: OK, what’s going on? The issue to understand is that when ‘rebasing’, Git first ‘grabs’ the master branch (or whatever branch you’re rebasing on) and then ‘applies’ all of your commits, one-by-one, *on top* of those commits. (As we said, this is main difference from ‘merging’, which integrates (well, merges) your commits in-between the commits on master.). Unfortunately, sometimes we have conflicts, but we can solve them.

Git has already told us, but to be sure a git status will show us the files being changed in *this* commit, including the conflicted ones. (Recall, we are rebasing all of the commits in our branch, one-by-one. Each commit might entail several files.)

You might conflicts in one or more files. Go into those files and fix the conflict[s], the standard way . Now back in the shell, we can proceed, recalling git’s instructions: git rebase –continue.

Note, you might have more than one conflicting commit – we are ‘rebasing’ all of your commits, so each one might contain a conflict you might have to solve.

4) Summary

Hopefully by now you have both understood what rebase does (specifically, differently form merging) and why it is important. Hopefully you have also lost your fear of rebase conflicts.

Remember that even if you don’t need it for yourself, the team as a whole can much better observe each other’s work if everybody rebases.

Be a team player – rebase.

OS X: Maximize windows the right way with RightZoom

OS X’s behavior regarding maximizing window size can be quite unpredictable and aggravating for those used to Window’s default maximize-to-screen. Shift/Command-clicking the maximize button doesn’t seem to get the job done consistently, either.

What you can use is RightZoom, a tiny utility to allow you to maximize your windows to fit the screen, and using the keyboard to boot!


1. Go to

2. Download RightZoom.

3. Open it by going to downloads folder, open folder, right-click on RightZoom, open, open.

4. Configure it to maximize your screen when pressing, say, Ctrl-Shift-M (I recommend using ‘Ctrl’, since it seems that with you can also toggle back to previous size).

5. Profit! Maximize screens to your heart’s delight.