January 24, 2015

How to Bulk-Import JSON to Postgres

The script above imports JSON into postgresql using the python library, pg8000. It scales, I just used it to import 589mb of JSON on a 1Gb netbsd virtual machine.

January 19, 2015

How to Find Out What's For Sale

The above code lists the latest for-sale items from craigslist. Demo run below:

Free Rice Cooker (san jose south): http://sfbay.craigslist.org/sby/zip/4853319800.html
LEATHER OFFICE CHAIR (santa clara): http://sfbay.craigslist.org/sby/zip/4842745192.html
Philips High Performance Amplified Indoor (san mateo): http://sfbay.craigslist.org/pen/zip/4853319255.html
free couches (glen park): http://sfbay.craigslist.org/sfc/zip/4853317431.html
Free hand truck. (santa rosa): http://sfbay.craigslist.org/nby/zip/4853308752.html
Free Rosemary bushes (pittsburg / antioch): http://sfbay.craigslist.org/eby/zip/4853309196.html
Extra Sturdy Old-School Wooden Desk (hayward / castro valley): http://sfbay.craigslist.org/eby/zip/4853305443.html
FREE QUEEN MATTRESS (oakland east): http://sfbay.craigslist.org/eby/zip/4853304845.html
Free IKEA MALM chest (santa clara): http://sfbay.craigslist.org/sby/zip/4853302532.html
My Wonderful free items for you (foster city): http://sfbay.craigslist.org/pen/zip/4841220459.html
FREE Mid Century 3 Piece Sectional by Starline Furniture | Sam Belz (oakland rockridge / claremont): http://sfbay.craigslist.org/eby/zip/4853300002.html
Free 6-foot folding table (glen park): http://sfbay.craigslist.org/sfc/zip/4853297053.html
Free Jeldwen Window (millbrae): http://sfbay.craigslist.org/pen/zip/4853294992.html
Free TV (west portal / forest hill): http://sfbay.craigslist.org/sfc/zip/4853292185.html
ENTERTAINMENT CENTER / Bookcase (rohnert pk / cotati): http://sfbay.craigslist.org/nby/zip/4853293213.html
FREE BOOK GIVEAWAY AFTER SALE (palo alto): http://sfbay.craigslist.org/pen/zip/4853288648.html
Evenflo Jenny Lind crib (millbrae): http://sfbay.craigslist.org/pen/zip/4853291170.html
Free landscaping rock (vallejo / benicia): http://sfbay.craigslist.org/eby/zip/4853290353.html
White Christmas lights (san rafael): http://sfbay.craigslist.org/nby/zip/4853288209.html
stuffs:junk, grease, photo, , jewel case, bike seat (san jose west): http://sfbay.craigslist.org/sby/zip/4853256207.html
Oak dining table 4 chairs and ikea buffet (cupertino): http://sfbay.craigslist.org/sby/zip/4853287277.html
sofa, chair and ottoman (mountain view): http://sfbay.craigslist.org/sby/zip/4853282364.html
sofa, chair and ottoman (mountain view): http://sfbay.craigslist.org/pen/zip/4853280610.html
Free A/V LCD TV Cart and Scientific Glassware (for you or schools or w (south san francisco): http://sfbay.craigslist.org/pen/zip/4853261265.html
Ikea Stuva Bench DRAWER (san jose downtown): http://sfbay.craigslist.org/sby/zip/4853276958.html

December 10, 2014

How to Expose Postgres Using REST

It's always a good idea to keep all unnecessary IP ports closed on a server. Indeed, there is a long list of them for every vendor (and if there isn't, it's not because the vendor is secure...). The code below lets you expose your database instance using webpy and pg8000, so it will run anywhere, including Windows. If I get a chance to enhance this, I'll be putting a help endpoint to the application.

December 5, 2014

How to Display a Multiplication Table Using Ruby

I needed to display a multiplication table of the first n prime numbers, where n is kept below 10. I chose ruby to solve the problem, because it has an implementation of the Sieve of Eratosthenes, out of the box(!!!). First the code:

And a sample run:

     2    3    5    7    11   13   17   19   23   29
     ---- ---- ---- ---- ---- ---- ---- ---- ---- ----
2  | 4    6    10   14   22   26   34   38   46   58
3  | 6    9    15   21   33   39   51   57   69   87
5  | 10   15   25   35   55   65   85   95   115  145
7  | 14   21   35   49   77   91   119  133  161  203
11 | 22   33   55   77   121  143  187  209  253  319
13 | 26   39   65   91   143  169  221  247  299  377
17 | 34   51   85   119  187  221  289  323  391  493
19 | 38   57   95   133  209  247  323  361  437  551
23 | 46   69   115  161  253  299  391  437  529  667
29 | 58   87   145  203  319  377  493  551  667  841

November 14, 2014

How to Listen to Podcasts

In the interest of using the best tool for the task, I've found that a simple zsh shell script is, by far, the easiest way to listen to podcasts, especially when used with the venerable mplayer:

November 11, 2014

How to Automate Your Rails Devops

This blog is all about automating my life, so that I can focus on what humans are good at and leave computers to focus on what they're good at. I live the dictum that figure it out once, automate it and forget about it. And by forget, I mean blog. I just added a mina autodeployment script for todo and I've decided to share my script below. So, without further ado, here we are:

# config/deploy.rb
require 'mina/bundler'
require 'mina/rails'
require 'mina/git'
# require 'mina/rbenv'  # for rbenv support. (http://rbenv.org)
require 'mina/rvm'    # for rvm support, like todo does. (http://rvm.io)

# Basic settings:
#   domain       - The hostname to SSH to.
#   deploy_to    - Path to deploy into.
#   repository   - Git repo to clone from. (needed by mina/git)
#   branch       - Branch name to deploy. (needed by mina/git)

set :domain, 'todo.d8u.us'
set :deploy_to, '/home/hdiwan/todos'
set :repository, 'git@bitbucket.org:hd1/todos.git'
set :branch, 'master'

# For system-wide RVM install.
#   set :rvm_path, '/usr/local/rvm/bin/rvm'

# Manually create these paths in shared/ (eg: shared/config/database.yml) in your server.
# They will be linked in the 'deploy:link_shared_paths' step.
set :shared_paths, ['config/database.yml', 'log']

# Optional settings:
#   set :user, 'foobar'    # Username in the server to SSH to.
#   set :port, '30000'     # SSH port number.
#   set :forward_agent, true     # SSH forward_agent.

# This task is the environment that is loaded for most commands, such as
# `mina deploy` or `mina rake`.
task :environment do
  # If you're using rbenv, use this to load the rbenv environment.
  # Be sure to commit your .rbenv-version to your repository.
  # invoke :'rbenv:load'

  # For those using RVM, use this to load an RVM version@gemset.
  invoke :'rvm:use[ruby-2.0.0@todo]'
  # Probably should use mina-scp, but what the hell, this works well.
  system 'scp ./db/development.sqlite3 hasan.d8u.us:todos/current/db/production.sqlite3'

# Put any custom mkdir's in here for when `mina setup` is ran.
# For Rails apps, we'll make some of the shared paths that are shared between
# all releases.
task :setup => :environment do
  queue! %[mkdir -p "#{deploy_to}/#{shared_path}/log"]
  queue! %[chmod g+rx,u+rwx "#{deploy_to}/#{shared_path}/log"]

  queue! %[mkdir -p "#{deploy_to}/#{shared_path}/config"]
  queue! %[chmod g+rx,u+rwx "#{deploy_to}/#{shared_path}/config"]

  queue! %[touch "#{deploy_to}/#{shared_path}/config/database.yml"]
  queue  %[echo "-----> Be sure to edit '#{deploy_to}/#{shared_path}/config/database.yml'."]

desc "Deploys the current version to the server."
task :deploy => :environment do
  deploy do
    # Put things that will set up an empty directory into a fully set-up
    # instance of your project.
    invoke :'git:clone'
    invoke :'deploy:link_shared_paths'
    invoke :'bundle:install'
    invoke :'rails:db_migrate'
    invoke :'rails:assets_precompile'
    invoke :'deploy:cleanup'

    to :launch do
      queue "mkdir -p #{deploy_to}/#{current_path}/tmp/"
      queue "touch #{deploy_to}/#{current_path}/tmp/restart.txt"

# For help in making your deploy script, see the Mina documentation:
#  - http://nadarei.co/mina
#  - http://nadarei.co/mina/tasks
#  - http://nadarei.co/mina/settings
#  - http://nadarei.co/mina/helpers

The only thing I have to do manually is restart the server.

November 4, 2014

How to Keep Track of Tasks #2

I did add multiple user support and updated the application serverside. The next few steps are to add full JSON support and allow for Google and Facebook login. The source code has been updated and checked in. The tl;dr of the changes is that devise was used for multiuser support.