How to Track ISP Uptime

The perl code below uses nmap (through the wonderful Nmap::Scanner module) and Text::CSV_XS -- though Text::CSV will work if you modify the import line:

use strict; use warnings; use diagnostics;

use Getopt::Long;
use Nmap::Scanner;
use Text::CSV_XS;

my $host = 'hasan.d8u.us';
GetOptions ('host=s' => \$host) or $host = 'hasan.d8u.us';
my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
my $last = 'up';
my $scanner = new Nmap::Scanner;
$scanner->register_port_found_event(\&host_found);
$scanner->register_no_ports_open_event(\&host_found);
do {
    $scanner->scan("-p 22 --system-dns -Pn $host");
    sleep 30;
} while 1;

sub host_found {
    my $self     = shift;
    my $host     = shift;
    my $port     = shift;
    my $time     = time();
    
    if ($host->status ne $last) {
        $csv->say($time, $host->status);
        $last = $host->status;
    }
}

How to solve fizzbuzz

I was on a phone screen today and was given the famous (or infamous) FizzBuzz problem. I will provide a strategy to solve the class of problems that fizzbuzz represents, namely, take two numbers, a and b both positive. The question is phrased thusly:

Generate numbers 1 to n, print "fizz" for every number divisible by a, "buzz" for every number divisible by b, and "fizzbuzz" for every number divisible by a*b. If not, print the number itself.

The key here is to start with the most restrictive case first, the a*b, before tackling the larger of a and b as an else if clause and the default case being the final line of the loop. Hence, in pseudocode:

for number in range(1, infinity):
if number % (a*b) == 0:
console_print 'fizzbuzz'
else if number % b:
console_print 'buzz'
else if number % a:
conosle_print 'fizz'
else:
console_print number


This code isn't in any language, but is probably easy-enough to translate to whatever language you need to. Also, it is general enough to (hopefully) enumerate the general case. I do, however, welcome further questions and feedback. Many thanks!

How to Add Grain in the Gimp

I have long wanted to learn how to retouch photos I've taken. One of the reasons I pay for Smugmug is that it won't mess with my creations. So, I took Terry's photos, adjusted its saturation, before finally adding grain. Here's how I did it:

• To allow for easy redo, you should start off by duplicating the layer, under the layers menu or by hitting CMD-SHFT-D. This will result in a new layer entitled "[filename] copy" with an identical thumbnail to the [filename] layer.
• Under the Colours menu, choose "Saturation".
• This will popup a dialog with options, check the "Split View" and adjust the line to the main part of the image to see how it will look.
• Set the saturation to 0. You'll have to tab away to see your results in the image.
• Save this setting for later use if you're happy with it.
• Now the image should look like the second one linked -- all colour removed. Have a cup of coffee
• Back now? Good... We will now add grain to the image. Under the "Filters" menu, choose "Noise" and "RGB Noise".
• Unclick "Independent RGB" and adjust the value to suit, watching the preview after tabbing away from the field.
• When you're done, you should have something looking like my third linked image.

How I Do Data Science

At a high level , a data scientist does one of 3 things. Given a time series, we want to determine one of three things:

1. What happened before?
2. What the values were in the middle?
3. What comes afterwards?

To estimate all three, data must be fit into an equation. I just set up a web service where you can post a delimited list of numbers, walk away, have it come up with histograms of the various fits and be notified by email when it's ready -- look for the fit tests for sample calls.

How to Predict the Stock Market?

I just completed a script that leverages two of my Json endpoints on units to make stock market predictions. The script is open-sourced on that gist and forms part of my test suite for the project.

How to Reverse-Geocode Addresses for Free

I just added functionality to units to reverse-geocode addresses. My test method for this code follows:

 @Test
 public void geocodeWorks() {

  Map<String, String> expected = new Gson().fromJson(
    "[{\"request\":\"2017 North Beverly Glen Blvd, 
Los Angeles, CA 90077, USA\",\"displayName\":\"Emerging Market Services, 2017, 
North Beverly Glen Boulevard, Beverly Glen, Westwood, Los Angeles, Los Angeles 
County, California, 90077, United States of America\",\"latitude\":\"34.1056855
\",\"source\":\"\",\"longitude\":\"-118.4461788\"}]",
    new TypeToken<List<Map<String, String>>>() {
    }.getType());
  StringMap actual = this.restTemplate.postForObject("/geocode",
    Collections.singletonMap("addr",
      "2017 North Beverly Glen Blvd, 
Los Angeles, CA 90077, USA"),
    StringMap.class);
  Assert.assertEquals(expected, actual);
 }

There's no rate limit on this API (for now), so use it to your heart's content. And you can view the up-to-date test code, as they evolve here.

'Staggering' New Data Shows Income Of Top 1% Has Grown 100 Times Faster Than Bottom 50% Since 1970

I found a time series regarding common dreams varying income classes share of GDP and visualized it, in an, I feel, more illustrative way than the original visualization. I'd appreciate your feedback on making my visualization clearer.

income  <- read.csv('/tmp/income.csv', header=TRUE, row.names=NULL)
colnames(income)  <- c('Year', '0-50pct', '50-90pct', '90-99pct', '99-99.9pct', '99.9-99.99pct', '99.99-100pct')
ggplot(income, aes(x=Year)) + geom_line(aes(y=income[,2]), colblue')

How to Script the Web For Free

This weekend, I decided to revive units functionality to parse HTML to JSON and put it up here. Will enhance it further shortly, by adding an error message and usage instructions. Knock yourselves out and let me know if you encounter any issues.

How to visualize a user's most Recent Subreddits

The above script lets you see which subreddits a given reddit user has been posting in over their last 100 submissions -- comments and posts. You'll need matplotlib, pandas, and requests to use it. Enjoy!

How to Use Gamlss with Ggplot2

I was exploring Gamlss and seeing if I could plot the resulting models with ggplot2. While a gamlss model is not, strictly speaking, coercible to data.frame, nor is there a fortify method included, I wrote a quick method and am providing it here, gratis: fortify.gamlss <- function(gamlss.obj) { return(data.frame(x=c(1:length(gamlss.obj$y)), y=gamlss.obj$y)) }

I would appreciate it if those responsible would incorporate this into the tree. Many thanks!