hur.st's bl.aagh

BSD, Ruby, Rust, Rambling

Comlim

Ruby command runner with resource limits

[ruby]

Comlim is a command builder for Ruby focusing on an easy interface to restricted execution—limiting memory, runtime, and command output.

Runner = Comlim.memory(32.megabytes)
               .cputime(30.seconds)
               .runtime(60.seconds)
               .output(4.kilobytes)

RubyRunner = Runner.command('ruby').arg('-e')
RubyRunner.arg('loop { }').execute # killed after 30 seconds
Continue Reading

Blooming Ruby

Prototypical BitArray and BloomFilter library

[ruby]

Yet another Ruby bloom filter, and a new bit array implementation, including a speedy JRuby-specific version.

This was extracted from an experiment in leaked password list processing, before I moved on to Golomb Compressed Sets, and might be worth extracting into a proper gem at some point.

Continue Reading

pkg-cruft

Help keep FreeBSD tidy

[ruby] [bsd]

A small Ruby script for helping deal with cruft on pkgng-based systems like FreeBSD.

Perhaps most interesting is the implementation of checkrestart, a feature similar to the Debian-goodies program of the same name, which can find running processes which may need restarting following an upgrade.

# pkg-cruft checkrestart
[MISSING EXECUTABLE] (tmux-2.7)? running as 17319 (tmux)
[MISSING EXECUTABLE] (zsh-5.5.1)? running as 20115 (zsh)
[MISSING EXECUTABLE] (weechat-2.2)? running as 36747 (weechat)
/usr/local/bin/mosh-server (mosh-1.3.2_4) running as 53815 (mosh-server)
Continue Reading

Reattempt

A jittery Enumerable retry and backoff library

[ruby]

Reattempt is a simple application of Ruby's Enumerators to provide a nice idiomatic interface to retries. At its most basic:

begin
  Reattempt::Retry.new.each do
    poke_remote_api
  end
rescue Reattempt::RetriesExceeded => e
  handle_repeated_failure(e.cause)
end
Continue Reading

mkpass

Command-line password generation

[rust]

mkpass is a simple command-line random password generator written in Rust, with built-in dictionaries to keep run-time dependencies to an absolute minimum.

Continue Reading

rtss

Relative Timestamps for Stuff

[rust]

rtss annotates its output with relative durations between consecutive lines and since program start.

Inspired by Kevin Burke's Golang tss, I thought it would be a fun exercise and a nice bit of Rust practice to implement something similar.

I'm fairly pleased with the result - it's considerably faster, running at hundreds of MB per second in my tests - and somewhat more featureful, including pty support, allowing it to work similarly to expect's unbuffer command.

Continue Reading

gcstool / ruby-gcs

Efficient set membership with Golomb Compressed Sets

[rust] [ruby]

Similar to Bloom Filters, Golomb Compressed sets allow for space-efficient probablistic storage of sets. In other words, you can ask a GCS if it's seen an object, and retrieve either "absolutely not" or "probably not" in response.

gcstool was my first Rust project, developed primarily to play about with the haveibeenpwned.com pwned-passwords-2.0.txt database. It can store all half a billion items with a false-positive rate of 1-in-50 million in just 1.6GB, importing them in just a few minutes, though with fairly high memory requirements.

Continue Reading

run-one

Friendly wrapper around lockf/flock, unofficial clone of an Ubuntu package

[shell]

Wanting to hone my shell scripting skills a bit, I ended up reimplementing a cute script by Dustin Kirkland.

run-one is basically a thin wrapper around lockf(1), or on Linux flock(1):

-% lockf /var/tmp/sleep-5.lock sleep 5
# <runs sleep 5 under the named lock>
-% run-one sleep 5
# <runs sleep 5 under a lock named by a SHA256 of its arguments>
Continue Reading

FlashFind

High performance multithreaded Find redux

[ruby]

Following the development of FastFind, I wanted to make something with a more pleasing API and better internal design.

The result is interesting, but still in need of work before I'd consider it for production use.

Continue Reading

FastFind

High performance multithreaded Find

[ruby]

FastFind is a drop-in replacement for the standard library Find package.

I wrote this for FreshBSD, to speed up scanning CVS repositories - walking 300,000 files takes a while, especially with an API that basically forces you to File.stat twice for each one.

Continue Reading

💾 Scandisk

Scan.co.uk Disk Price List

[ruby] [web]

Sick of Scan.co.uk being a pain in the ass to browse for this sort of thing, I wrote a scraper and frontend to browse available storage devices and their prices.

Unauthorised, unofficial, and unsupported.

🖩 Bloom Filter Calculator 🖩

It has, like, graphs, and stuff.

[php] [javascript] [web]

Bloom filters are one of my favourite data structures, in no small part because of just how simple they are.

I first encountered them when I worked at Newzbin, when we needed a way of quickly determining if we'd seen a Usenet Message-ID before.

Their simplicity meant we were very quickly able to go from concept to a stable general-purpose microservice which served us well for many years.

One of the trickier aspects is knowing how to size one - typically you know how many items and what sort of false-positive rate you can stand, but how do you translate that to how many bits you need and how many hash functions to use?

Well, bam. The latest iteration of my calculator can calculate almost any valid set of parameters, and give you pretty graphs in real time, while offering server-side fallback. I hope you find it useful.

pqsort

Fast partial quicksort

[c]

pqsort is the sorting library I wrote for Newzbin's custom search engine - it's a lightly modified quicksort capable of partitioning and sorting only part of a list.

Quicksort lends itself very well to this kind of use due to how it works - it naturally partitions the list based on a pivot, and so it's trivial to simply ignore any pivot side which cannot contain the desired results.

Continue Reading

k8temp

Command-line AMD K8 CPU thermal diode reader

[c] [bsd]

A foray into lower-level code, bashing PCI registers and using a bit of custom assembler, k8temp offers a command line interface to the thermal sensors on AMD's old K8 CPUs.

Continue Reading

FreshBSD

BSD Commit Log Search

[ruby] [bsd] [web]

A search engine/viewer for source code commits, focused on projects of interest to the BSD community.

Currently into its fourth rewrite, backed by Elasticsearch, PostgreSQL and Redis, with the site itself running on JRuby and Roda, making use of concurrent-ruby to parallelise data retrieval and inserts.

Previous versions have used Rails, Sinatra, Padrino, Solr, MySQL, and memcached.

php-serialize

PHP's serialize()/unserialize() in Ruby

[ruby] [php]

Many, many years ago, I wrote this for unclear reasons. I don't think I actually used it for anything. It implements the PHP serialize() and unserialize() functions in Ruby, allowing you to read/write PHP sessions and otherwise share objects between the two.

Now the various forks of it have nearly half a million downloads between them. php-serialize is the "official" version, with nearly half of those.

The exact incept date is uncertain, the earliest mention of it a quick search finds is ruby-talk:73669, 2003-06-16.