How to Implement Basic Authentication in Flask

 if not request.headers.get("Authorization"):                                                                                                                                                                   
            return jsonify({"error": "Need auth header"})
        auth_hdr = base64.urlsafe_b64decode(
            request.headers.get("authorization").replace("Basic ", "")
        ).decode()
        username, password = auth_hdr.split(":")
        with open("flatFileForPasswords") as fin: 
            reader = csv.reader(fin)
            for r in reader:
                if r[0] == username:
                    if r[1] != password:
                        return jsonify({"error": "Wrong auth info"})

For when I forget how to do this.

How to add a Row to a Dataframe in Pandas

This is a triivial task that, ought not to warrant its own blogpost, but it took me almost 24 hours to figure out, so here we are: df.loc[len(df)] = np.nan

How to Forecast Time Series in R

I was pointed to caretForecast to machine learning algorithms for time series data forecasting and decided to post this as a tutorial on how I did it (and before I forget). We'll use the UK Unemployment rate from FRED as our data. Without further ado, then:

require(caretForecast)
require(caret)
require( quantmod)
i <- 8
retail.wide <- getSymbols('UNRTUKA', auto.assign=F, src='FRED')
dtlist <- caretForecast::split_ts(as.ts(retail.wide), test_size = 12)
training_data <- dtlist$train
testing_data <- dtlist$test
fit <- caretForecast::ARml(training_data, max_lag = 12, caret_method = "glmboost", 
            verbose = FALSE)
fc <- caretForecast::forecast(fit, h = length(testing_data), level = c(80,95))
caretForecast::accuracy(fc, testing_data)

  Your output will look like:
  

    initial_window = NULL. Setting initial_window = 225
> >                        ME     RMSE       MAE       MPE     MAPE      MASE
Training set 7.265863e-16 1.353875 0.9509708 -12.93998 25.39738 0.9920345
Test set     1.349505e+00 1.832183 1.4355957  18.10500 19.86195 1.4975859
                  ACF1 Theil's U
Training set 0.1524037        NA
Test set     0.6623072  1.997636

But, what we're really interested in are the forecasted values, which are embedded in fc, so let's have a look:

> fc
    Point Forecast    Lo 80    Hi 80    Lo 95    Hi 95
246       4.745729 3.340558 6.150899 2.168297 7.323160
247       4.766436 3.361266 6.171607 2.189005 7.343867
248       4.797869 3.392698 6.203040 2.220438 7.375300
249       4.860259 3.455089 6.265430 2.282828 7.437690
250       4.937558 3.532388 6.342729 2.360127 7.514990
251       5.013973 3.608803 6.419144 2.436542 7.591405
252       5.094964 3.689794 6.500135 2.517533 7.672395
253       5.174285 3.769114 6.579456 2.596854 7.751716
254       5.237659 3.832488 6.642829 2.660228 7.815090
255       5.298847 3.893677 6.704018 2.721416 7.876279
256       5.361814 3.956644 6.766985 2.784383 7.939245
257       5.416543 4.011372 6.821714 2.839112 7.993974
  

Why the GRA and why now?

I caught up on George this morning, while debating which cheese to devour (said debate continues as I write this piece). His rant midweek was about the Scottish National Party's Gender Reassignment Act.

My thesis rests on several planks, the first of which is that the loyal opposition is more about loyalty to the power brokers and less about opposing their policy prescriptions, especially along pocketbook issues, like adjusting economic levers. Even foreign policy, which realistically speaking should not be the primary concern of the homeless who litter the streets of the United States and Britain, is a realm of policy where the two sides of Congress or Parliament are one.

However, every now and again, there are elections, where a number of candidates have to differentiate themselves. So, if your voting records are 99 percent identical, how do you do it? There are certain differences which we cannot (yet) gloss over. Namely, that I have brown skin and Mr Galloway does not. Or, perhaps, it is that Mrs Harris is a woman and Mr Pence is not.

In other words, elections circa 2022 are fought on social issues, when most working class voters are concerned with economic issues.

Boundary commissions have made it so that the working class doesn't have much say in elections. And confronted with the choice of one candidate who promises to bomb Kyiv and the other to bomb Mariupol, it's no wonder they choose the third option -- opt out of the system altogether.

How to Make ChatGPT Produce Images with a Transparent Background

Friday, I ran into TalkAnimate founder Leslie Pound in Los Angeles and played with chatGPT's image search. We found it was too clever-by-half, in that when we asked for an image on a transparent background, it found on with how the photo editing packages render transparent backgrounds -- ie the checkerboard background. Well, I was.able to render the image with a truly transparent background and I've been able to replace the background with a transparent version, just alter the text value in the URL. Leave a comment if you encounter any trouble.

How to Simulate Turnout in Tunisia's Constitutional Referendum Using R and the Tidyverse

On coming home, I looked at R-bloggers, while vegetating over the Giants game. I have a keen interest in Middle Eastern politics and specifically, the machinations in North Africa's Maghreb region.. So, it is with keen interest that I read that Tunisia is having a constitutional referendum and that he's run a few simulations of the referendum. I've copied the run below:
require(tidyverse)
require(ggthemes) # we need these two and our man forgot to include them.

N <- 7000000

N -> number.of.voters # for clarity purposes

stations <- 4500
vote_assign <- sample(1:stations,number.of.voters,replace=T, prob=sample(1:3,stations,replace=T))
# loop over turnout, sample polls, estimate turnout

over_turnout <- parallel::mclapply(seq(.05,.5,by=.1), function(t) {
# polling station varying turnout rates

station_rates <- rbeta(n=number.of.voters,t*20,(1-t)*20)

# randomly let voters decide to vote depending on true station-level turnout rate

pop_turnout <- lapply(1:stations, function(s) {
tibble(turnout=rbinom(n=sum(vote_assign==s), size=1,prob = station_rates[s]), station=s)}) %>% bind_rows

over_samples <- lapply(1:1000, function(i) {

# sample 100 random polling stations 1,000 times
sample_station <- sample(1:stations, size=50)

turn_est <- mean(pop_turnout$turnout[pop_turnout$station %in% sample_station])

return(tibble(mean_est=turn_est, experiment=i))
}) %>% bind_rows %>%
mutate(Turnout=t)

over_samples
},mc.cores=10) %>% bind_rows over_turnout_biased %>%
group_by(Turnout) %>%
summarize(pop_est=mean(mean_est), low_est=quantile(mean_est,.05), high_est=quantile(mean_est, .95)) %>%
ggplot(aes(y=pop_est,x=Turnout)) +
geom_pointrange(aes(ymin=low_est, ymax=high_est),size=.5,fatten=1) +
geom_abline(slope=1,intercept=0,linetype=2,colour="red") +
theme_economist() + # I prefer this to theme_tufte, as Robert employed
theme(text=element_text(family="")) +
labs(y="Estimated Turnout",x="True Turnout", caption=stringr::str_wrap("Comparison of Mourakiboun estimated (y axis) versus actual turnout (x axis). Red line shows where true and estimated values are equal. Based on biased samples of 50 polling stations with higher turnout stations more likely to be sampled. However, simulation assumes no problems with recording votes.")) +
ylim(c(0,0.5)) +
xlim(c(0,0.5))
From our experience in dealing with Brexit, referenda tend to be very hard to predict if the country isn't accustomed to having them regularly. And, Robert does acknowledge this.

How I Hire

I've interviewed dozens of people for many positions. Some I waved through, a handful I rejected at every one of my various employers. I've had a few cry out of frustration at not being able to solve the problem, I've had a handful who've been resourceful and found the answers on the web in real time. I'm here to tell readers that knowing how to factor a polynomial (or not, as the case may be) is not the reason candidates are asked questions like "what is 2 to the 32nd power?" or "Why are manhole covers round?".

If you think that grinding on LeetCode or doing hackerrank perfectly is the end-all, be-all of your interview preparation, you will not be bumped ahead. If you feel that your graduating from valedectorian at Choate entitles you to a job, you are not going to get ahead.

Now, what will and, more importantly, why will it get you the job and keep you there?

We'll start with the "why" first. So, you went to Stanford. Congrats, you are one of 16,914 individuals who go to Stanford every year. What makes you deserving over any of them? You have a 4.0 GPA, what does that tell me? It tells me that you can give the grader what they want to see for 4 years straight. While your skills at reading people's intentions are commendable, how do I know that you'll continue to do this for us? You need to convince me (along with every other person) that you are the sort of person whose name I will look forward to seeing daily in my inbox -- you probably can gather where this is going...

What I want to know is "what makes you unique?" and not just nother hapless Cardinal, who found me on a list of alumni and reached out because we pay well enough to let you afford a monthly trip to a gentlemen's club. Do you like to travel? Was your home destroyed by a bomb in 1986? Did you grow up without parents? How did you get interested in this sort of work? You want me to remember you as a person, so that, when I look at your CV in a few days, the fact that you were the Stanford graduate that skiied Whistler at age 5 pops into my head (or whatever it is).

How to Track ISP Uptime

The perl code below uses nmap (through the wonderful Nmap::Scanner module) and Text::CSV_XS -- though Text::CSV will work if you modify the import line:

use strict; use warnings; use diagnostics;

use Getopt::Long;
use Nmap::Scanner;
use Text::CSV_XS;

my $host = 'hasan.d8u.us';
GetOptions ('host=s' => \$host) or $host = 'hasan.d8u.us';
my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
my $last = 'up';
my $scanner = new Nmap::Scanner;
$scanner->register_port_found_event(\&host_found);
$scanner->register_no_ports_open_event(\&host_found);
do {
    $scanner->scan("-p 22 --system-dns -Pn $host");
    sleep 30;
} while 1;

sub host_found {
    my $self     = shift;
    my $host     = shift;
    my $port     = shift;
    my $time     = time();
    
    if ($host->status ne $last) {
        $csv->say($time, $host->status);
        $last = $host->status;
    }
}

How to solve fizzbuzz

I was on a phone screen today and was given the famous (or infamous) FizzBuzz problem. I will provide a strategy to solve the class of problems that fizzbuzz represents, namely, take two numbers, a and b both positive. The question is phrased thusly:

Generate numbers 1 to n, print "fizz" for every number divisible by a, "buzz" for every number divisible by b, and "fizzbuzz" for every number divisible by a*b. If not, print the number itself.

The key here is to start with the most restrictive case first, the a*b, before tackling the larger of a and b as an else if clause and the default case being the final line of the loop. Hence, in pseudocode:

for number in range(1, infinity):
if number % (a*b) == 0:
console_print 'fizzbuzz'
else if number % b:
console_print 'buzz'
else if number % a:
conosle_print 'fizz'
else:
console_print number


This code isn't in any language, but is probably easy-enough to translate to whatever language you need to. Also, it is general enough to (hopefully) enumerate the general case. I do, however, welcome further questions and feedback. Many thanks!

How to Add Grain in the Gimp

I have long wanted to learn how to retouch photos I've taken. One of the reasons I pay for Smugmug is that it won't mess with my creations. So, I took Terry's photos, adjusted its saturation, before finally adding grain. Here's how I did it:

• To allow for easy redo, you should start off by duplicating the layer, under the layers menu or by hitting CMD-SHFT-D. This will result in a new layer entitled "[filename] copy" with an identical thumbnail to the [filename] layer.
• Under the Colours menu, choose "Saturation".
• This will popup a dialog with options, check the "Split View" and adjust the line to the main part of the image to see how it will look.
• Set the saturation to 0. You'll have to tab away to see your results in the image.
• Save this setting for later use if you're happy with it.
• Now the image should look like the second one linked -- all colour removed. Have a cup of coffee
• Back now? Good... We will now add grain to the image. Under the "Filters" menu, choose "Noise" and "RGB Noise".
• Unclick "Independent RGB" and adjust the value to suit, watching the preview after tabbing away from the field.
• When you're done, you should have something looking like my third linked image.