Easy cluster parallelization with ZeroMQ

Easy cluster parallelization with ZeroMQ

TL;DR: I design short wrapper scripts to send batch jobs to a cluster. The first architecture (ventilator-worker-sink) is suboptimal, therefore I consider another pattern to dispatch the jobs efficiently. Recently I had to make a big chunk of computations…

Transform parts of your screen into Mario levels with Screentendo

Transform parts of your screen into Mario levels with Screentendo

You know what's a good cure for office boredom? A Mac app that changes parts of your screen into a playable level straight out of Super Mario Bros. Thanks to Aaron Randall's Screentendo desktop add-on, a similar process to that of taking a screenshot...

Google announces Literata, its new e-book typeface

Google has announced a new typeface for Google Books: Literata. Replacing Droid Serif as the default typeface for e-books via Google Books, Google says Literata is “perfect for long reads on all devices”. Created in conjunction with TypeTogether, Google…

H2O – an optimized HTTP server with support for HTTP/1.x and HTTP/2

Copyright (c) 2014-2019 DeNA Co., Ltd., Kazuho Oku, Tatsuhiko Kubo, Domingo Alvarez Duarte, Nick Desaulniers, Marc Hörsken, Masahiro Nagano, Jeff Marrison, Daisuke Maki, Laurentiu Nicola, Justin Zhu, Tatsuhiro Tsujikawa, Ryosuke Matsumoto, Masaki TAGAWA,…

Measuring page weight

Measuring page weight

It used to be so easy. Measuring the weight of a web page in the early days of the web was merely a matter of waiting for the page to finish loading and then counting up the size of its constituent resources. Since then, however, a lot has changed. The…

Using BitTorrent with Amazon S3

BitTorrent is an open, peer-to-peer protocol for distributing files. You can use the BitTorrent protocol to retrieve any publicly-accessible object in Amazon S3. This section describes why you might want to…

Dnsdiff

Dnsdiff

dnsdiff compares the responses between two nameservers and provides output in universal diff format. Run dnsdiff before delegating authority of a zone to a new provider to ensure that the new provider has imported your zonefile correctly. usage: dnsdiff…

Watermark.js: Watermarking for the browser

A functional library for watermarking images in the browser. Written with ES6, and made available to current browsers via Babel. Supports urls, file inputs, blobs, and on-page images. Note: For anyone that is interested: I ported this to a ClojureScript…

How 500px serves up 500TB of high res photos

How 500px serves up 500TB of high res photos

By Paul Liu, DevOps Developer, at 500px. You can find him here on StackShare and on Twitter 500px is an online community for premium photography. Millions of users all over the world share, discover, sell and buy the most beautiful images. We value…

A Tmux crash course: tips and tweaks

A Tmux crash course: tips and tweaks

If you are one of those devs who uses the terminal a lot and ends up with way too many tabs open, or practices pair programming, then this post is for you. During the last months, I’ve started using Tmux a lot. Since I’ve found it to be very useful, I…

The Failure of Agile

I am proud to be one of the 17 founders/authors of the The Agile Manifesto back in 2001. I think it provided a jolt of energy, hope of a better way of doing things, of creating software and making the world work better. It was a pivotal turning point. But…

Scaling Secret: Real-Time Chat

Scaling Secret: Real-Time Chat

I try a lot of iOS apps and many offer chat features (especially social networking and dating apps). Almost all are unique in their own ways and nearly all of them are broken in one way or another, usually they’re either too slow (not real-time) or too…

This drone follows you down the trail after you toss it up in the air

This drone follows you down the trail after you toss it up in the air

Aerial footage is a nice way to capture those action sports endeavors, and it's even better if you can fit the gear in your backpack. Lily ticks those boxes, and all you have to do to launch it is toss it up in the air. Once airborne, the camera UAV ...

Large-scale cluster management at Google with Borg

an interesting/influential/important paper from the world of CS every weekday morning, as selected by Adrian Colyer Large-scale cluster management at Google with Borg – Verma et al. 2015 Borg has been running all of Google’s workloads for the last ten…

Kubernetes is going to support rkt

For many, Docker is the next big thing in cloud computing. But some big names—most notably Google—are now backing an alternative to this enormously influential technology. In December, one of Docker's earliest supporters, the Silicon Valley startup…

Generated (Virtual) Columns in MySQL 5.7 (labs)

About 2 weeks ago Oracle published the MySQL 5.7.7-labs-json version which includes a very interesting feature called “Generated columns” (also know as Virtual or Computed columns). MariaDB has a similar feature as well: Virtual (Computed) Columns. The…

Indexing 101: Optimizing MySQL queries on a single table

I have recently seen several cases when performance for MySQL queries on a single table was terrible. The reason was simple: the wrong indexes were added and so the execution plan was poor. Here are guidelines to help you optimize various kinds of single…

Boolean parameters to API functions considered harmful (2011)

one line of code at a time Can you spot the bug in the following code? _xhr.open(options.type, options.url, options.sync); The problem is that the last param to xhr.open is whether the xhr is async. Ouch. Turns out, this code was written intentionally,…

8 reasons to use 1Password that don't involve storing passwords

8 reasons to use 1Password that don't involve storing passwords

Available for Mac, iOS, Windows, and Android, 1Password is a must-have for desktop and mobile users seeking equal parts online security and convenience. But there’s far more to this software than its single-purpose name might imply. 1Password owners…

SSLMate – Buy SSL certs from the command line

Secure your website the easy way. SSLMate makes it easy to buy, deploy, and manage your SSL certs. Get SSL certificates from the command line in under 60 seconds. No complicated openssl commands or copy-and-pasting certificate bundles.…

pquery binaries with statically included client libs now available!

pquery binaries with statically included client libs now available!

After we released pquery to the community, and as we started logging bug reports with pquery testcases, it quickly became clear that pquery binaries with statically compiled-in client libraries would be of great convenience, both for ourselves and for the…

Mac 911: How to erase your Mac's hard drive the right way

Mac 911: How to erase your Mac's hard drive the right way

When you sell or donate a Mac or give it to a family member, it’s best to make a clean break: wipe the puppy clean, reinstall the latest version of OS X, and hand off a system that you’re not worried has remnants of yourself on it. But how can you be sure?…

Why Netflix is embracing Python over Java

Why Netflix is embracing Python over Java

Netflix is increasingly turning to Python over Java to power certain aspects of its video-streaming service, such as generating and processing alerts, boosting resilience, securing transactions, producing deployable AMIs (Amazon Machine Images), and for…

Show HN: Kaiwa, a Modern Open-Source XMPP Web Client

Show HN: Kaiwa, a Modern Open-Source XMPP Web Client

Inspired by the best and built for XMPP, Kaiwa is a beautiful web-based client for the only standardized chat protocol. Combined with Prosody, one of the best XMPP servers out there, Kaiwa provides an unparalleled XMPP experience, on par…

Introducing Vector: Netflix's On-Host Performance Monitoring Tool

Introducing Vector: Netflix's On-Host Performance Monitoring Tool

Vector is an open source host-level performance monitoring framework, which exposes hand-picked, high-resolution system and application metrics to every engineer’s browser. Having the right metrics available on demand and at a high resolution is key to…

Google open-sources its Santa Tracker a few months late

Google open-sources its Santa Tracker a few months late

Google might want to improve its timing a bit. You see, the search giant has released the source code for its Santa Tracker apps... in April. In other words, right when the snow has finished melting in many corners of the world. That's a little too l...

Power tips for managing files in Apple’s Finder

Power tips for managing files in Apple’s Finder

While some of OS X’s applications have their own data structures—like Mail and Photos—Finder is likely where you will be doing most of your file organization. After you have created or collected hundreds (if not thousands) of files on your Mac, you might…

Experimenting with HHVM at Etsy

Experimenting with HHVM at Etsy

In 2014 Etsy’s infrastructure group took on a big challenge: scale Etsy’s API traffic capacity 20X. We launched many efforts simultaneously to meet the challenge, including a migration to HHVM after it showed a promising increase in throughput. Getting our…

The cost of not properly managing your databases

The cost of not properly managing your databases

Every day hundreds of millions of dollars are wasted by allowing improperly tuned or misconfigured systems, misunderstood infrastructure, and inefficient IT operations to live and thrive in data centers around the globe. There are both direct and indirect…

The best cross-platform writing apps for Mac and iOS

The best cross-platform writing apps for Mac and iOS

iOS devices have completely changed the way we write. Not only has iOS given us the flexibility to quickly jot down our thoughts wherever and whenever they strike, it's fundamentally flipped our expectations of the humble text editor. In an effort to…

Get your own USB PID

Welcome to pid.codes. If you’re a maker, hobbyist, or startup company producing your own USB device, you’ve probably discovered that you need a USB Vendor ID and Product ID to uniquely identify your device to computers. The USB-IF’s position is that the…

A Git Style Guide

This is a Git Style Guide inspired by How to Get Your Change Into the Linux Kernel, the git man pages and various practices popular among the community. Translations are available in the following languages: If you feel like contributing, please do so!…

Why Netflix Chose NGINX as the Heart of Its CDN

Why Netflix Chose NGINX as the Heart of Its CDN

In the few years since its introduction, Netflix’s online video streaming service has grown to serve over 50 million subscribers in 40 countries. We’ve already shared some of the best practices that Netflix’s software development engineers adopted as they…

Java VM Options You Should Always Use in Production

This post is a cheatsheet with enumeration of options, which should be always used to configure Java Virtual Machine for Web-oriented server applications (i.e. Web Front-End) in production or production-like environments. For lazy readers full listing is…

Show HN: Fast autocomplete service written in Go and JavaScript

Download autocompeter.min.css and autocompeter.min.js and insert into your site. Submit your page information to like this: curl -X POST -H "Auth-Key: yoursecurekey" -d url=/some/url -d title="Some Title" https://autocompeter.com/v1 Or by…

Instagram gives Apple Watch developers a hand

Instagram gives Apple Watch developers a hand

With the Apple Watch set for release on April 24, third-party application developers are rushing to write extensions that can run on the device. Now Instagram is sharing some code it wrote for its own app to help others. The Facebook-owned company has…

Scaleway: Bare metal SSD ARM servers

Object Storage is now available in the Paris FR-PAR region. Discover it now 🇫🇷 By clicking or navigating this website site, you agree to allow our collection of information on Scaleway to offer you an optimal user experience and to keep track of…

Hello World in EC2 Container Service

EC2 Container Service (ECS) is a new AWS deployment option for Docker containers. It was launched at the AWS Re-Invent conference in November 2014. Its still in a limited preview and its not yet in the AWS web console so you have to use the AWS CLI. The…

Simple Script for Getting a Device’s UDID

When I need to grab a device’s UDID, it has always felt heavy to me to have to launch iTunes or Xcode just to get a simple 40-character string. After years of sighing about it, I finally did something. Below is a simple bash script that uses OS X’s…

Why is this song used in so many GoPro videos?

Why is this song used in so many GoPro videos?

February 23rd, 2012. Electronic violinist Lindsey Stirling uploads the official music video for "Crystallize" to YouTube. Two days later, user "riley lux" uploads a video titled "DH long boarding on a windy day." In it, a group of friends enjoy some…

Companies with a flat structure are weird

Specifically, companies with a flat structure (no managers at all) are weird. Decades of management tradition tell us that Someone must be in charge and make sure everyone’s doing their work. Otherwise, everyone will goof off, right? And yet, somehow…

Gg – Git Shortcuts, Colored Outputs, and More

Gg – Git Shortcuts, Colored Outputs, and More

gg helps you work with git more efficiently, saving you keystrokes for your most prized projects. Think of gg as a wrapper for the git commands that you run all the time; a wrapper that adds functionality and is aesthetically pleasing. There's more to…

Ask HN: Designing Useful Dashboards

I've set myself a side project to scrape and display car hire price data, to compare suppliers and trends. I've never done this before and my original goal was to play & learn new skills - but the challenge has become displaying this information in a…

Performance Tuning Linux Instances on EC2

Performance Tuning Linux Instances on EC2

17 Jan 2018 » How To Measure the Working Set Size on Linux home At the last AWS re:Invent, I gave a talk on "Performance Tuning EC2 Instances", where I showed how my team (Performance and Reliability Engineering) tunes Linux EC2 instances at Netflix.…

Chalk – Colorize your terminal output

Chalk – Colorize your terminal output

* Used by ~40,000 packages as of March 1, 2019 $ npm install chalk const chalk = require('chalk'); console.log(chalk.blue('Hello world!')); Chalk comes with an easy to use composable API where you just chain and nest the styles you want. const chalk =…

The best practices of HTTP1 are harmful in a HTTP2 world

HTTP2 will mean a change in how we should build websites. The best practices of HTTP1 are harmful in a HTTP2 world. HTTP1 is slow and inefficient for the majority of today's use cases on the web. HTTP1.x is the version of HTTP we are all familiar with.…

Milo Casagrande: Make nginx Return a JSON Response

This is more of a “write here so I do not forget” kind of post. I set out trying to make a simple “maintenance” mode for a REST API (that speaks only JSON) without having to handle it in the code base. nginx has a really cool support for maintenance mode…

Analyzing the Patterns of Numbers in 10M Passwords

Analyzing the Patterns of Numbers in 10M Passwords

The primary purpose of a password is to serve as an unique verification identifier for a given user. Ideally, the password for a given website or service should be both random and unique; if the letters and/or numbers in the password follow any patterns,…

Custom UX checklist for your project

Integration of all of the above Symbian, Qt, Meego, Android, Windows Phone 7 (the new Lumia phones are 7.5), iPhone, iPad Also, we will be ready to move on to Windows Phone 8 when it's released early next year or so. We can work very smoothly with…

GitLab Annex: Large Binaries with Git

GitLab Annex: Large Binaries with Git

WARNING git-annex support on GitLab EE was deprecated in GitLab 8.17 (2017/02/22), and was permanently removed in GitLab 9.0 (2017/03/22). Read through the migration guide from git-annex to git-lfs. Updated on 2017/04/04. The biggest limitation of git…

My First LLVM Compiler

A walkthrough of writing a basic compiler with LLVM. No prior experience assumed. In a short space of time, I was able to go from zero C++ knowledge, and no experience with LLVM, to a fully-fledged compiler. It’s a lot of fun, let me show you how! Our…

Email Regex that works 99.99%

Just copy and paste the email regex below for the language of your choice. Feeling hardcore (or crazy, you decide)? Read the official RFC 5322, or you can check out this Email Validation Summary. Note there is no perfect email regex, hence the 99.99%. Try…

DIY: kernel panic OS X and iOS in 10 LOC

DIY: kernel panic OS X and iOS in 10 LOC

After receiving quite a few reports from users of www.frida.re of kernel panic upon attaching the second time to a process, I finally got around to debugging the kernel to figure out what was going on. First, a little background. Frida hooks function…

PHP Static Analysis with HHVM and Hussar

Wayfair Engineering places special emphasis on software testing as a means of maintaining stability in production. The DevTools team, which I am a member of, has built and integrated a number of tools into our development and deploy process in order to…

Major Changes from Solr 4 to Solr 5

Some of the major improvements in Solr 5 include: * New, and restructured, example Config Sets available by default * SolrJ now has first class support for the Collections API * Implicit registration of /replication, /get, and /admin/* request handlers …

How to identify a font, any font, anywhere you find it

With tens of thousands of fonts in circulation, it’s tough to identify a specific font from memory, when you see it. Fortunately, you don’t have to—there are free websites, and an iOS app, that can identify font samples for you on the fly. The process is…

The load-balanced capture effect

I'm going to describe a common frontend/backend split design. Try to visualize it and see if it resembles anything you may have run, built, or relied on over the years. Then see if you've encountered the "gotcha" that I describe later on. Let's say…

Facebook Now Runs on Networking Gear Designed by Its Own Engineers

Look out, Cisco. Facebook is now running on computer networking gear designed by the company’s own engineers. According to Najam Ahmad, who oversees network engineering at Facebook, the company is using its own designs inside at least some of the data…

Teletext for Raspberry Pi

Teletext for Raspberry Pi

This software generates a teletext signal in software. No hardware mods are needed. This support both PAL (Teletext) and NTSC (CEA608 format captions). Have a Raspberry Pi connected to a TV by composite video. It doesn't matter if you are running X or…

How to reset your Mac's NVRAM, PRAM, and SMC

How to reset your Mac's NVRAM, PRAM, and SMC

When your Mac starts acting up, you’ll probably run through some common troubleshooting procedures, such as restarting it, running Disk Utility, and perhaps performing a Safe Boot. Your repair repertoire should also include a couple of additional…

Top 7 Reasons Developers Contribute to Open Source Projects

Why on earth would a developer want to contribute to an open source project? We ran a study on this a little while ago¹. The reasons are all quite simple, and of course are different for many people. We can broadly categorize the type of contributor into…

[$] FreeNAS — network-attached storage with ZFS

Thus far, this series has looked at Linux distributions that are optimized for network-attached storage (NAS) deployments. This installment will take a slightly different turn: the system under review (FreeNAS) is indeed a free distribution for NAS…

Webseclab – Web security test cases and a construction toolkit

Webseclab – Web security test cases and a construction toolkit

Webseclab contains a sample set of web security test cases and a toolkit to construct new ones. It can be used for testing security scanners, to replicate or reconstruct issues, or to help with investigations or discussions of particular types of web…

CloudPing – Estimate the latency from your browser to each AWS region

Amazon Web Services™ are available in several regions. Click the button below to estimate the latency from your browser to each AWS™ region. Amazon Web Services and AWS are trademarks of Amazon.com, Inc. or its affiliates in the United States and/or…

Montage the world from the command-line with Google and Instagram

Montage the world from the command-line with Google and Instagram

Montage the world from the command-line with Google and Instagram All the menial JSON parsing we've had to do…there is a reason for the suffering (though maybe not an end to it): if we can read and access this nearly ubiquitous data format, then we have…

Datapath.io: Provider-Independent Elastic IP Addresses

All your multi-cloud applications connected via a secure and dedicated private networkGlobal SD-WAN enabled Network Backbone with PoPs all over the worldReplace and widen the scope of your legacy MPLS with simple and intuitive connectivity across cloud…

GitLab Pages (similar to what GitHub offers)

It requires Jekyll to be installed. GitLab Pages will build Jekyll sites or simply copy content from non-jekyll sites to publicPagesDir. Create a SSH key pair for GitLab to use as deploy keys. ssh-keygen -t rsa -C "[email protected]" -f…

How to design your own great-looking business cards

How to design your own great-looking business cards

Few design projects are as nerve-wracking—and important—as designing your own business card. Just like the clothes you wear, your business card tells the recipient if you’re professional, artistic, or a big ol’ ball of cheese. Aside from the aesthetic…

Visualizing AWS Storage with Real-Time Latency Spectrograms

How 6 of the world’s largest companies use Kub + Sysdig. Having worked for several years in the wireless networking space, I always found spectrum analyzers like this one extremely useful. I always found such a representation of the wireless spectrum not…

Show HN: Get your local and public IP addresses in JavaScript

Show HN: Get your local and public IP addresses in JavaScript

Firefox and Chrome have implemented WebRTC that allow requests to STUN servers be made that will return the local and public IP addresses for the user. These request results are available to javascript, so you can now obtain a users local and public IP…

The Secrets of Street Names and Home Values

The Secrets of Street Names and Home Values

IN “Romeo and Juliet,” the young Miss Capulet poses one of literature’s most famous questions: “What’s in a name?” When it comes to a street name, the answer is: a lot. Street names tell stories. They tell us if a neighborhood is expensive or affordable,…

Using Percona Cloud Tools to solve real-world MySQL problems

Using Percona Cloud Tools to solve real-world MySQL problems

For months when speaking with customers I have been positioning Percona Cloud Tools (PCT) as a valuable tool for the DBA/Developer/SysAdmin but only recently have I truly been able to harness the data and make a technical recommendation to a customer that…