issue #35: cgroups, sqlmap, etcd3, Elasticsearch, Charles, Packer & more!July 3, 2016 - Mattias Geniar
Welcome to cron.weekly issue #35 for Sunday, July 3rd, 2016.
Once again, a packed edition! Sorry about last week, it seems using smilies and emoji’s in newsletter isn’t always the best plan. I learned my lesson, I’ll never smile again.
Enjoy this week’s issue!
And if you have a spare moment, I’d love to hear from you: what do you like, dislike, recommend, … If you like the newsletter, share it with friends & colleagues, it’s a great kick to see the newsletter grow in size every week!
Nike has open sourced a few projects. Mostly developer-oriented now, with frameworks for parsing JSON and tracing in Java, but it’s a very cool move from an otherwise rather close company. Can’t wait to see what else they release!
NASA also open sources many of their code and projects. Quite a few more than the Nike example, as you can tell by the scrollbar on the site.
Inspired by a presentation at VelocityConf, this post further dives into the roles of a “DevOps” vs a “Site Reliability Engineer” (SRE), a term coined by Google. Many arguments for skillsets to have when working at scale vs. skillsets most organisations are actually looking for, because they aren’t Google-sized.
All of Github’s data is now also available in Google’s BigQuery tables, allowing interesting queries to be run over all of Github’s open source projects.
This is a fun read for everyone that’s had the pleasure of maintaining a popular open source project. It describes all the emotional steps involved: being embarrassed to open source your code, frustration, anxiety for the burden of maintaining the project, …
This is a proposal to take the concepts of cgroups on Linux, to limit and control resources, to the web. Soon, web browsers may get the same primitives that control CPU, memory, network and I/O as we do on Linux.
The Suse team made another music cover: this time it’s The Beatles with “Come Together”. In case you missed it, a few months earlier they did a fabulous parody with Uptime Funk (“Don’t reboot it, just patch”)!
Tools & Projects
The sqlmap project turned 10 years last week, so a well-deserved mention here: sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers. In short, it’s a scanner you can launch at any website (with permission!) and it’ll find SQL injection vulnerabilities.
This is a fun tool to learn and teach about encryption ciphers: by using an emoji substitution algoritme, you can “encode” and “decode” messages. Looks like a good way to teach kids about ciphers, too.
etcd is a distributed, consistent key-value store for shared configuration and service discovery (like Consul), and the CoreOS team has just released version 3.0! The post highlights the efforts involved in moving from etcd2 to etcd3, the differences, how concurrency works and much more.
Tired of paying Evernote as a document/note syncer? Turtl is a self-hosted, open source alternative. It lets you take notes, bookmark websites, and store documents for sensitive projects, all on your own hardware.
If you run a VMware environment, this driver allows your Docker containers to consume vSphere storage (like vSAN, VMFS or NFS) directly. If you want to know more about Docker’s storage drivers, I recommend the podcast I recorded a few weeks ago about Docker.
Charles is an HTTP proxy / HTTP monitor / Reverse Proxy that enables a developer to view all of the HTTP and SSL / HTTPS traffic between their machine and the Internet. This includes requests, responses and the HTTP headers (which contain the cookies and caching information).
Guides & Tutorials
A promising series! The first part describes the terminology, master vs. data nodes, the storage model of Elasticsearch and how basic CRUD (Create, Read, Update & Delete) operations work.
As a sysadmin, it’s your job to be as efficient as possible. Papertrail allows you to easily give developers or colleagues access to relevant logs to help them troubleshoot. No API or coding required, just an easy-to-customise URL, and they can see everything they need. (Sponsored)
A good reminder that you can use the .profile file to customise your shell upon login, even replacing the shell altogether.
A good guide on password management, monitoring login attempts, SSH ports and the like.
This post describes some of the problems of Docker containers and introduces Habitat, a new tool to manage dependencies, with a “docker layer” on top of it. It keeps the Docker layers clean and focuses the container on just the bare minimum.
Many of these principles apply to other Linux distro’s too. A very good introduction to using the ‘perf’ tool and framework to troubleshoot performance issues on Linux.
This post introduces Ansible, Packer, Vagrant and containers to build “immutable” servers.
With many of the sysadmins tasks moving to automation and config management, it becomes more and more important to structure the code you write and make it as readable and maintainable as possible. Whether that’s Ansible, Chef, Puppet or something else, this post has good examples of “easy to read code” you can inherit.
A step-by-step guide on getting multiple certificates issued using Let’s Encrypt (the free Certificate Authority) for use in Nginx.
This is a very good explanation of “lock files” in Linux and in which different forms they can come. Since a lot of tools use lock files, this is a good basic set of knowledge any Linux sysadmin should have.