How does printf really work?

printf is magical. Did you ever stop and ask yourself how it works?

Contrary to most functions, it accepts a variable number of arguments, and somehow transforms them into a formatted string! The GNU code of printf is pretty simple:

printf (const char *format, ...) {
va_list arg;
int done;
va_start (arg, format);
done = vfprintf (stdout, format, arg);
va_end (arg);
return done;
}

if you look closely, it uses a weird ... syntax, performs a couple of va_ calls and one vfprint call.

to understand printf, we first need to understand how va_ works, then move to printf.

If you’re ready for some hard-core c and assembly, start by reading how va_ works!

Read More

May 2017 Major Outage

The blog suffered a major outage today - it was offline for around six hours. It took me around 90 minutes to get it working once I had time to do so.

Why

A few days ago I created a new droplet for a homebrew scraping project I’ve been working on lately.

Today I decided its time to throw it, and pressed the big, red, destroy button. Then I noticed I deleted the wrong droplet and accidentally deleted my blog!

Disaster Recovery Plan

I had daily backups setup already (remember Poor mans daily blog backups?) which were almost up to date.

All I needed to do is to add a few updates to the latest post. Fortunately, every time I post to LinkedIn, they cache my posts at “oded-ninja.cdn.ampproject.org/c/s/oded.ninja/…”

Up until now I kind of hated that cache. Every time I updated a blog post, it took forever to refresh, which was a real pain. Only this time I was actually grateful that cache existed.

Anyway, have you ever heard of the AMP Project?

The AMP Project is an open-source initiative aiming to make the web better for all. The project enables the creation of websites and ads that are consistently fast, beautiful and high-performing across devices and distribution platforms.

Ghost has pre-baked AMP support, Which I’ve set up to automatically redirect mobile clients.

Snapshots

DigitalOcean provide a droplet snapshot service for only 20% of the cost of the virtual server.

The problem is that snapshots wouldn’t help me at this point, because they get deleted with the droplet. Good thing I created that backup service.

Restore

I forked Ghost a few months ago, and instead of testing my changes on production, I put all my configuration in a private git repository.

That proved really useful for local testing, but especially for restoration. I had a habit of checking changes on up-to-date backups, which meant I restored the blog locally every few days.

Plan Execution

  1. Create a new Droplet
  2. Create new Read-only Deploy Keys
  3. Clone the repository, and build it.
  4. Restore the backup from Google Drive
  5. Pause Cloudflare
  6. Check the website is working
  7. Setup Lets Encrypt
  8. Resume Cloudflare
  9. Re-create my Keybase website proof.

Issues I Encountered

  1. I forgot to update the new droplet’s IP at namecheap, my DNS provider.
  2. When checking the website, before turning on SSL, I kept getting redirected to the https endpoint. That’s because it was saved in the browsers HSTS set. Fix? clear my browser’s HSTS settings
  3. Chrome has an internal DNS cache which needed to be refreshed. Fix? flush DNS records and sockets at chrome://net-internals/#dns and chrome://net-internals/#sockets respectively.
  4. I didn’t read the Lets Encrypt setup instructions thoroughly and got blocked for an hour after several failed attempts to set it up. Conclusion? RTFM!
  5. I forgot to update my website proof on Keybase.

Lessons Learned

First of all, DON’T PRESS THE BIG, RED, DESTROY BUTTON BEFORE MAKING SURE YOU ARE REMOVING THE RIGHT DROPLET!

Second, I need to make a few adjustments -

  • Enhance my backup script to include nginx’s configuration as well.
  • Setup a mechanism to perform backup after every update, or at least reduce the interval between backups.
  • Automate recovery steps, or at least document them. When I’m under pressure (or drunk), I forget steps :|

Conspiracy Theory: Intel's AMT Vulnerability & The Ken Thomson Hack

Around two weeks ago Intel announced a critical privilege escalation bug that was laying around its Active Management Technology (AMT) login page for the past seven years. The exploit allows a remote attacker to take control of vulnerable devices with ease.

I’ve read many posts that mock the programmer who introduced it, and the (lacking) testing framework and processes to make sure such things never happen.

But, what if no one made a mistake, and the whole thing is a result of an elaborate hack?

  • How much can you trust software?
  • Have you ever checked the validity of the sources your acquire your software from?
  • Can you trust your own code? Have you ever checked the tooling that compiles or runs it?

In 1984, Ken Thompson, a known figure in the hacker community and one of the authors of UNIX, proposed we can’t. In his remarkable paper, Reflections On Trusting Trust, Ken outlines a hack that many considers the worst hack imaginable: The Ken Thomson Hack.

This blog post is a bit long (but worth it!) and made out of three parts:

  1. The AMT Vulnerability
  2. The Ken Thomson Hack
  3. How 1 & 2 lead to a mega conspiracy

! Disclaimer: The conspiracy theory is completely made up.

Interested? Awesome. Start by reading about the AMT vulnerability.

Read More

Ultimate Guide to Winning a Hackathon

I keep running into people that tell me they’re unqualified to go to Hackathons, because their coding skills aren’t good enough. This post is for everyone who wants to win a Hackathon, and specifically to people who avoid them.

I recently participated and won the biggest Hackathon in Israel. I love Hackathons, and I love winning too. Getting a cash reward is fun, but not as much as winning!

After reading the above, you probably think I’m extremely competitive and cocky, but your’e missing the point - my definition for winning a Hackathon is probably different than yours.

Read More

Nuclear Gandhi & Binary Arithmetic

Nuclear Gandhi is the nickname given to the Indian historical figure Mahatma Gandhi as portrayed in the turn-based strategy video game series Civilization.

A bug in the game caused Gandhi, who is a known pacifist in real life, to turn into a nuclear-obsessed maniac that made India the most hostile civilization in the game.

The cause was a glitch in the artificial intelligence settings for Gandhi’s aggression level: Gandhi started with the lowest level of aggression to reflect his historical legacy of pacifism: 1.

When a player adopted democracy in Civilization, their aggression would be automatically reduced by 2, which means that Gandhi’s aggression level should have gone to -1, but instead the aggression level went all the way up to 255, making him as aggressive as a civilization could possibly be.

Interesting right? but how the heck does -1 become 255?

A bit of math

Don’t worry. I’m not going to dive in too deep. There’s a plethora of blog posts and explanations on how integer arithmetic & representation work.

I’ll explain just enough in order for you to understand what’s going on.

Integer representation

510{5_{10}} in 8-bit binary is000001012{000001012}, pretty straight forward.
But what about510{-5{10}}? How is it implemented? lets draft a possible solution.

First, we need to know the sign of the number. We’ll reserve the most significant bit for the sign, and use the rest as the values. Second, We’ll make sure we don’t break compatibility and set the sign bit for positive numbers to zero, and negative numbers to one. In this scenario a
signed 8-bit number would range from -127 to 127.

Now, in our hacky system,510{5{10}} won’t change, and510{-5{10}} will be100001012{10000101_2}.

But here’s the catch - regular arithmetic doesn’t work:

510+510=000001012+100001012=100010102=10100{5{10}} + {-5{10}} = {00000101_2} + {10000101_2} = {100010102} = {-10{10}} \ne 0

We can build custom assembly arithmetic, but that’s an over-kill.

Two’s complement

Two’s complement is a mathematical operation on binary numbers, as well as a binary signed number representation based on this operation. Its wide use in computing makes it the most important example of a radix complement. - Wikipedia

TL;DR: a different system that makes arithmetic work as you’d expect.

// 00000101
int x = 5;
// ~x = 11111010
// ~x + 1 = 11111011
int negativeX = ~x + 1;

For example, addition of510{5{10}} and510{-5{10}} works like we expect:510+510=000001012+111110112=000000002{5{10}} + {-5{10}} = {00000101_2} + {11111011_2} = {00000000_2}

More information is out of scope for this blog post. If you’re interested, start from the answers for What is “2’s Complement”? on StackOverflow.

Ok, so what happened?

A Civilization’s aggression level was saved as an unsigned char, which can’t represent negative values.

Gandhi’s aggression level started at110{1{10}}, and when democracy arrived, it was reduced by two:110210=000000012000000102=000000012+111111102=111111112{1{10}} - {2_{10}} = {00000001_2} - {00000010_2} = {00000001_2} {\color{red} +} {11111110_2} = {11111111_2}

if the aggression level variable was signed, then the binary would be interpreted as110{-1{10}}, which is what we’d expect. Instead, it was unsigned, which means it got interpreted as25510{255{10}}.

… And Gandhi turned from a pacifist into a warmonger: “Greating from M. Gandhi, ruler and king of the Indians… Our words are backed with NUCLEAR WEAPONS!”

Do you REALLY understand floating points?

I’m going to ask you a couple of questions. If you answer all of them correctly, and understand why - good job! this post is not for you.

Otherwise, If you’re a normal human being, consider reading this post. It’ll save you hours of useless debugging. Honestly, If the engineering who built Ariane V read it (and set their compiler to warning as error) their rocket wouldn’t blow up.

What’s the answer? yes or no?

float x = 0.7;
printf(x == 0.7 ? "yes" : "no")

What will be printed?

float x = 4 / 3;
printf("%f", x);

What’s the answer? yes or no?

float x = 1.0/3.0;
double y = 1.0/1234567.0;
printf(((x+y) - x) == y ? "yes" : "no");

Are both lines equal?

float x = 0.20;
double y = 0.20;
printf("%4.20f\n", x);
printf("%4.20f\n", y);

Now that I’ve got your attention, lets go over the answers real quick. Once you get to the end of this blog post, You’ll understand them fully and be able to impress your coworkers with useless knowledge.

float x = 0.7;
printf(x == 0.7 ? "yes" : "no")

output: no.

float x = 4 / 3;
printf("%.3f", x);

output: 1.000

float x = 1.0/3.0;
double y = 1.0/1234567.0;
printf(((x+y) - x) == y ? "yes" : "no");

output: no.

float x = 0.20;
double y = 0.20;
printf("%4.20f\n", x);
printf("%4.20f\n", y);

output:

0.20000000298023223877
0.20000000000000001110

Read More

1st place @ HackIDC 2017

Yesterday, after 24 hours of hard work, zero sleep & a lot of fun, We won first place at HackIDC!

[The team by order of appearance: Carmel Rabinovitz, Nadav Eliyahu, Amir Livne, Inbal Ben Yehuda & Me]

What is HackIDC

HackIDC is Israel’s leading student Hackathon, held annually at IDC Herzliya. It is a great opportunity to build something new and exciting, together with coding and design enthusiasts. Students work in teams of up to five people for 24 hours to create a web, mobile, or hardware product.

What we did

Our team crafted a smart bracelet for conferences that swaps contact details between participants using a regular handshake. The match is done by monitoring acceleration data gathered from the bracelet’s Red Amber chip, and processing it in real-time by a learning algorithm that we developed.

The data that we collected could help conference organizers gain tremendous knowledge about the participants, and that’s only the beginning. We’re planning on continuing the project by adding more insightful features.

! All the sensor related tools were written in C++, MatLab helped us make sense of them. The data crunching part was written in python, and the website using WiX.

[The prototype we used during development]

Electra Challenge

HackIDC introduced a new “Challenge” section this year. Some of the sponsors had their own challenges for the teams: Bank Leumi, Murata & Electra.

We decided to participate in Electra’s challenge in parallel to our “main” solution - hook into an air conditioner unit to provide alerts to operation teams when interesting events occur:

  • A unit installation failed
  • Compressor gas had reached a critical point

We extracted all the sensor data straight from the unit, sent it to a server at the backend that crunched it, and raised alerts when they matched a pattern.

! The entire project was written in python. The frontend used flask, bootstrap and JS.

[The website showed real alerts raised by the data that was simulated by Electra’s air condition unit]

switch is much more than a glorified if

this post is a bit low level. I’ve been writing a lot of C & C++ recently, and found a few concepts eye opening. This post is dedicated to all the people that think that switch is a glorified if.

I’m going to do my best to avoid writing a lot of assembly, and make this post as easy as possible. Don’t worry if you don’t know any assembly, I’ll explain everything!

Read More