pozorvlak: (Default)
Wednesday, February 3rd, 2016 10:34 pm
Following on from my 2015 review post, I should document my New Year's Resolutions for this year. In descending order of priority:
  1. Make a first ascent in the Greater Ranges. Same plan as last year: we've found some 4000m mountains in Kyrgyzstan with no recorded ascents, we've applied for financial assistance from the Mount Everest Foundation (who exist to fund this sort of thing), I've booked the time off, I'm learning Russian via a combination of night classes and Duolingo (befriend me here!), and I'm training in earnest. Speaking of which:
  2. Get my body mass below 70kg, from a starting point of 81.2kg on New Year's Day. I want to retain a few kilos of body fat, because food is heavy and burning fat on a route is way better than burning muscle, but every extra gram of body mass will have to be carried 2000m up a mountain, at altitude, as fast as possible. If you've never done hard physical work at altitude while overweight, let me share a secret with you: it is Not Fun. I'm tracking my weight using the Libra app for Android, which implements Hacker's Diet-style smoothing on your noisy daily weigh-in data; calorie intake via MyFitnessPal; and calorie expenditure via a FitBit exercise-tracking band, because MyFitnessPal's calorie-per-hour estimates for most forms of exercise are laughably high. FitBit can sync calories-burned to MFP, which I currently have set up; Libra doesn't sync to either of them, which is annoying, but I really want the trend rather than the raw weight data. FitBit also have a native food-tracking system, so I may ditch MFP at some point.
  3. Show up for work in a timely fashion. This is something I struggle with horribly at the moment. I almost always arrived in time for our morning standup meeting at my last job, but now I'm working remotely as part of a distributed team, and we don't have any equivalent for that. I've just signed up for Beeminder and created a "Do Less" goal with units of "minutes late to work", and a fairly generous weekly target; we'll see how that goes.
  4. Actually do some work while I'm there. Not sure how to make this SMART or how to achieve it. The Pomodoro technique is... moderately effective, if I actually start doing it (which is much easier if I show up not-too-late in the morning). RescueTime integrates with Beeminder, so I could set myself a goal for "spend more time looking at an IDE, terminal or job-related websites" (or one for "spend less time looking at blogs and social media"). By the way, a pet peeve: if you're reading about the Pomodoro Technique and thinking "sounds interesting, but 25 minutes isn't enough" then you are not the kind of person who needs it. 25 minutes is a major challenge for some of us.
  5. Read an average of one book a week. You can follow my progress on this one at Goodreads.

pozorvlak: (Default)
Tuesday, January 26th, 2016 06:10 pm
I got into an interesting conversation on Twitter last night about "singular 'they'" (the use of the pronoun "they" to refer to only one person), in which a number of people expressed strongly-held opinions at me. However, I think "singular they" covers several closely-related but different constructions, some more common than others, and it's worth teasing them out.
  1. An abstract person whose gender is unknown, or a person chosen from a set containing people of both genders. "If the reader does not like my use of singular 'they', they are invited to stick it up their jumper".
  2. A concrete person whose gender is unknown. "We wish to thank Reviewer 2 for their critical feedback & sincerely apologize for not having written the manuscript they would have written".
  3. An abstract person whose gender is known: "Just because somebody's feminine, it doesn't mean they have a vagina".
  4. A concrete person whose gender is known, but which you do not want to reveal. "I met someone, and they make me feel so happy... that someone is a guy".
  5. A concrete person whose gender is known, but does not fit within the male/female binary: "Jamie is agender; they'd prefer that you use the pronouns 'they', 'their' and 'them' when talking about them".
  6. A concrete person whose gender is known, binary, and obvious from context, but which you do not want to encode in the pronoun. "Tom gave me their shirt". This was the sense @cassolotl was originally asking about.
As far as I can remember, I've been using senses 1 and 2 all my life. Usage 3 hadn't occurred to me before I started reading Language Log entries on singular 'they', and I doubt I'd even have picked up on it as notable if I'd encountered it in the wild. Usage 4 feels like a mild stretch. I only became aware of nonbinary people in the last few years; usage 5 still feels weird to me, but since it only causes me mild (and decreasing) annoyance and apparently can save the person concerned considerable distress, I'm happy to use it when that's what they prefer (there are also a number of invented pronouns like "zie", which some nonbinary people prefer over "they" - again, I'm happy to use those pronouns if the person concerned prefers it). Usage 6 trips my internal "this utterance is wrong" alarm. Even one of my interlocutors from last night, who was extremely bullish on singular 'they', told me they would not use it in sense 6.

I think it's interesting that in my idiolect, singular 'they' carries an implication of abstraction or uncertainty: compare my reactions to senses 3 and 6. I'm not claiming that my personal feelings of grammatical acceptability are high-quality linguistic data, but they should probably carry more weight than a set of formal rules that were inferred originally from Latin.

A popular argument for singular 'they' is the argument from history: as one Twitter interlocutor told me, "Singular 'they' goes back to the 14th Century. Chaucer, Shakespeare, Austen and Shaw can't be wrong". But arguments from history have limited validity when applied to modern English. In the opposite direction, consider the following passage from the novel Tinker, Tailor, Soldier, Spy, published in 1974:
They had been following the coast path, each lost in his thoughts: she to Haydon, he supposed, he to Control...
That's a clear usage of "he/his" as a gender-neutral pronoun in sense 1 above. Does that seem right to you? It feels weird bordering on outright wrong to me, and yet it was apparently considered correct usage only forty years ago. And if Le Carré's constructions can pass from correct to incorrect in so short a time, so can Austen et al's. Chaucer, in particular, should not be considered a reliable guide to modern English usage. But also, while I'm sure you can find examples of Shakespeare and Austen using singular 'they' in senses 1 and 2, can you find examples of them using it in senses 4-6, or even 3? I suspect not, but would welcome correction.

So, before telling me that singular 'they' is completely correct in all situations and only idiots object to it, please stop and ask yourself if you'd use it without qualm in all the senses I list above. A straw poll among a left-leaning group of friends gave me only one outright "yes", with the rest offering variations on "sense 6 is weird, but acceptable". Interestingly, there was a disagreement about whether invented gender-neutral pronouns such as "zie" were acceptable in sense 4, a situation for which I'd have thought they were custom-designed. My own view is that singular 'they', while out of fashion for a long time, is enjoying a renaissance and that all the uses I list above will be considered completely normal within a couple of decades: even though I find some of them odd, I might as well get used to them now.
pozorvlak: (Default)
Monday, January 11th, 2016 08:26 pm
 So, you have irritable bowel syndrome?

Yep, since about 2000. My symptoms are mostly low-grade diarrhoea, flatulence, bloating and fatigue, but occasionally I get abdominal pain, or rumbling noises in my abdomen (for which I recently learned the delightful technical medical term “borborygmi”).

Sounds annoying, but not life-changingly awful.

That’s about right. Also embarrassing, and frequently unpleasant for anyone downwind of me.

And you’re on a new diet for your IBS?

Yep, the "low-FODMAP" diet.

Never heard of it. What’s the teal deer?

I have to cut out gluten, dairy, pulses, onions and garlic.


And some other things like mushrooms, most fruit, cauliflower, pistachios, cashews and artichokes. And I’m meant to limit my intake of butternut squash, but not other squashes. There are a whole bunch of rules.

You said you have to cut out onions and garlic - how about leek?

Yep. And shallots. And the white parts of spring onions, but the green parts are OK.

And you’re not allowed dairy?

Yep - no milk, cream or ice cream for me. But I’m allowed hard cheese. And Brie and Camembert.

OK, I call shenanigans. You’re making this up.

I’m really, really not. I was told to follow this diet by my doctor.

Oh, your “doctor”, eh? Are they a doctor of crystal Reiki homeopathy? Because this is some “astral quantum chakra”-level enterobollocks.

No, he’s a medical doctor, in a perfectly ordinary NHS surgery. Though he did tell me that he didn’t know much about FODMAPs before handing me a printout and sending me on my way.

OK, supposing - just supposing - you’re not making this up, what the hell are FODMAPs?

Fermentable Oligosaccharides, Disaccharides, Monosaccharides And Polyols.

Say what?

Short-chain carbohydrates which are poorly absorbed by the small intestine, but which make the bacteria in my large intestine go OM NOM NOM DELICIOUS CARBS, causing my enteric nervous system to go “Oh dear! I appear to be full of nasty fermentation products. Better get rid of them quickly!”, and in turn causing me to go “Bloody hell, tummy, I’ve already been six times today and I have a meeting in five minutes, do we really have to do this right n- OK, I guess we do. Dammit, I’d just washed those trousers.”

But gluten is a protein.

Well spotted. The problem is actually with wheat, which has high levels of fructans (one of the two classes of oligosaccharides under the FODMAP label; the other class is galacto-oligosaccarides, which are common in pulses). However, supermarkets and restaurants have “gluten-free” labelling, not “low-fructan” labelling.

Is there any evidence for the FODMAP theory?

Yes! There are peer-reviewed publications demonstrating both the mechanism described above and the benefits of a low-FODMAP diet for patients with IBS.

Is this a revolutionary new treatment, discovered by a mom, that doctors hate?

As previously mentioned, I was told to follow it by my doctor, and AIUI the mechanism described above was already the established one for lactose intolerance (lactose being the disaccharide in the acronym). The FODMAP concept appears to have been devised by Dr Sue Shepherd, a dietician and academic. I don’t know if she has any children.

Is it helping?

It seems to be, yes! I’ve been on the diet for about a month now, and my symptoms are definitely reduced - my bad days now are about as bad as my average days before, and my good days are almost normal. And lapses from the diet seem to be followed by a couple of bad days.

And you think that this may be connected to your utter failure at vegetarianism a few years ago?

Possibly. When I cut out animal protein from my diet, I started eating a lot more pulse-based dishes to compensate, and most of those also contained onion and/or garlic. When I restored meat to my diet, my intake of pulses and alliums went down. That could explain some of the horrible fatigue, lethargy and brain-fog I experienced for those two years, and the dramatic improvement I experienced when I started eating meat again. This theory predicts that my symptoms will be triggered by either fructans or galacto-oligosaccharides, and I'll find that out soonish.

When I saw you last you said you could/couldn’t eat Foodstuff X, but now you’ve changed your mind! What gives?

It’s possible that I was wrong the first time - this is a complicated diet with a lot of rules, I don't always remember them all, and it’s not always obvious what chemicals are in a given food. For instance, lots of surprising foods contain onion powder, which is a major no-no. Or it’s possible that I’ve moved onto a different phase of the diet - in the first two months I’m meant to eliminate all FODMAPs as far as possible, and then over the next few months I’m meant to slowly re-introduce the different FODMAP classes one at a time to see which ones actually cause me problems. Anyway, I realise that this diet makes me a major pain to feed, so thank you for making the effort!

pozorvlak: (Default)
Thursday, December 31st, 2015 08:18 pm

Last year I made three New Year's Resolutions:

  1. Get better at dealing with money.
  2. Run a marathon.
  3. Make a first ascent in the Greater Ranges.

Number 2 was an obvious success: I finished the Edinburgh Marathon in 4:24:04, and raised nearly £900 for the Against Malaria Foundation. I'd been hoping to get a slightly faster time than that, but I lost several weeks of training to a chest infection near to the end of my training programme, so in the end I was very happy to finish under 4:30. The actual running was... mostly Type II fun, but also much less miserable than many of my training runs, even at mile 21 when I realised that literally everything below my navel hurt. Huge thanks to everyone who sponsored me!

Number 3 was an equally obvious failure. My climbing partner and I picked out an unclimbed mountain in Kyrgyzstan and got a lot of the logistics sorted, but then he moved house and started a new job a month before we were due to get on a plane to Bishkek. With only a few weeks to go and no plane tickets or insurance bought yet (and them both being much more expensive than we'd expected - we'd checked prices months earlier, but forgot how steeply costs rise as time goes on), we regretfully pulled the plug. We're planning to try again in 2016 - let's hope all the good lines don't get nabbed by Johnny-come-lately Guardian readers.

Number 1 was a partial success. I tried a number of suggestions from friends who appear to have their financial shit more together than me (not hard), but couldn't get any of them to stick. I was diagnosed with ADHD at the end of 2014; I don't want to use that as an excuse, but it does mean that some things that come easily to most people are genuinely difficult for me - and financial mismanagement is apparently very common among people with ADHD. The flip-side, though, is that I have a license to do crazy or unusual things if they help me be effective, because I have an actual medical condition.

I've now set up the following system:

  • my salary (minus taxes and pension contributions) is paid into Account 1;
  • a couple of days later, most of it is transferred by standing order into Account 2;
  • all bills are paid from Account 2 by direct debit, and Account 2 should maintain enough of a balance for them to always clear;
  • money left in Account 1 is available for spending on day-to-day things;
  • if I pay for something on a credit card, I pay it off from Account 1 (if small) or Account 2 (if big) as soon as possible;
  • Account 2 pays interest up to a certain ceiling; above that I'm going to transfer money out into a tax-efficient Account 3, which pays less interest but which doesn't have a ceiling.

I'll have to fine-tune the amount left in Account 1 with practice, but this system should ensure that bills get paid, I can easily see how much money I have left to spend for the month, and very little further thought or effort on my part is required.

While I was in there, I took the opportunity to set up a recurring donation to the Against Malaria Foundation for a few percent of my net salary - less than the 10% required to call yourself an Official Good Person by the Effective Altruism movement, but I figure I can work up to it.

It's too early to say whether the system will work out, but setting it up has already been a beneficial exercise - before, I had seven accounts with five different providers, most of them expired and paying almost zero interest (in one file, I found seven years' worth of letters saying "Your investment has expired and is now paying 0.1% gross interest, please let us know what you want us to do with it.") I now have only the three accounts described above, from two different providers, so it should be much easier to keep track of my overall financial position. Interest rates currently suck in general, but Accounts 2 and 3 at least pay a bit.

I've also started a new job that pays more, and [profile] wormwood_pearl's writing is starting to bring in some money. We're trying not to go mad and spend our newfound money several times over, but we're looking to start replacing some broken kit over the next few months rather than endlessly patching things up.

What else has happened to us?

I had a very unsuccessful winter climbing season last year; I was ill a lot from the stress of marathon training, and when I wasn't ill the weather was terrible. I had a couple of good sessions at the Glasgow ice-climbing wall, but only managed one actual route. Fortunately, it was the classic Taxus on Beinn an Dothaidh, which I'd been wanting to tick for a while. I also passed the half-way mark on the Munros on a beautiful crisp winter day in Glencoe.

One by one, my former research group's PhD students finished, passed their vivas, submitted their corrections, and went off, hopefully, to glittering academic careers or untold riches in Silicon Valley. Good luck to them all.

In June, I did the training for a Mountain Leadership award, the UK's introductory qualification for leading groups into the hills. Most of the others on the course were much fitter than me and more competent navigators, but the instructor said I did OK. To complete the award, I'll need to log some more Quality Mountain Days and do a week-long assessment.

In July, we went to Mat Brown's wedding in Norfolk, and caught up with some friends we hadn't seen IRL for far too long. Unlike last year, when it felt like we were going to a wedding almost every weekend, we only went to one wedding this year; I'm glad it was such a good one. Also, it was in a field with camping available, which really helped to keep our costs down.

In July, I started a strength-training cycle. I've spent years thinking that my physical peak was during my teens, when I was rowing competitively (albeit badly) and training 15-20 hours a week, so I was surprised to learn that I was able to lift much more now than I could then - 120kg squats versus around 90kg (not counting the 20kg of body weight I've gained since then). Over the next few weeks, I was able to gain a bit more strength, and by the end I could squat 130kg. I also remembered how much I enjoy weight training - so much less miserable than cardio.

In August, we played host to a few friends for the Edinburgh Fringe, and saw some great shows, of which my favourite was probably Jurassic Park.

In September, we went to Amsterdam with friends for a long weekend, saw priceless art and took a canal tour; then I got back, turned around within a day and went north for a long-awaited hiking trip to Knoydart with my grad-school room-mate. There are two ways to get to Knoydart: either you can take the West Highland Line right to the end at Mallaig, then take the ferry, or you can get off at Glenfinnan (best known for the viaduct used in the Harry Potter films) and walk North for three days, sleeping in unheated huts known as bothies. We did the latter, only it took us six days because we bagged all the Munros en route. I'm very glad we did so. The weather was cold but otherwise kind to us, the insects were evil biting horrors from Hell, and the starfields were amazing. It wasn't Kyrgyzstan, but it was the best fallback Europe had to offer.

In October, I started a new job at Red Hat, working on the OpenStack project, which is an open-source datacenter management system. It's a huge, intimidating codebase, and I'm taking longer than I'd like to find my feet, but I like my team and I'm slowly starting to get my head around it.

That's about it, and it's five minutes to the bells - Happy New Year, and all the best for 2016!

pozorvlak: (Default)
Wednesday, May 13th, 2015 06:24 pm

Over the seven years or so I've been using Git, there have been a number of moments where I've felt like I've really levelled up in my understanding or usage of Git. Here, in the hope of accelerating other people's learning curves, are as many as I can remember. I'd like to thank Joe Halliwell for introducing me to Git and helping me over the initial hurdles, and Aaron Crane for many helpful and enlightening discussions over the years - especially the ones in which he cleared up some of my many misunderstandings.

Learning to use a history viewer

Even if all you're doing is commit/push/pull, you're manipulating the history graph. Don't try to imagine it in your mind - get the computer to show it to you! That way you can see the effect your actions have, tightening your feedback loop and improving your learning rate. It's also a lot easier to debug problems when you can see what's going on. I started out using gitk --all to view history; now I mostly use git lg, which is an alias for log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr)%Creset' --abbrev-commit --date=relative, but the effect is much the same.

The really important lessons to internalise at this stage are

  • Git maintains your history as a graph of snapshots (you'll hear people using the term DAG, for "directed acyclic graph").
  • Branches and tags are just pointers into this graph.

If you understand that, then congratulations! You now understand 90% of what's important about Git. Just keep that core model in mind, and it should be possible to reason about the rest.

Relatedly: you'll probably want to graduate to the command-line eventually, but don't be ashamed to use a graphical client for any remotely unfamiliar task.

Understanding hash-based identifiers

You know those weird c56ab7f identifiers Git uses for commits? Those are actually serving a very important role. Mark Jason Dominus has written a great explanation, and I also suggest having a poke around in the .git/objects directory of one of your repos. MJD talks about "blobs" and "trees" which are also identified with hashes, but you don't actually have to think about them much in day-to-day Git usage: commits are the most important objects.

Learning to use git rebase

This shouldn't really qualify as a GLUE - my very first experience with Git was using the git-svn plugin, which forces you to rebase at all times. However, even if you're not interoperating with SVN, you should give rebase a go. What does it do, you ask? Well, the clue's in the name: it takes the sequence of commits you specify, and moves them onto a new base: hence "re-base". See also the MJD talk above, which explains rebasing in more detail.

Note to GUI authors: the obvious UI for this is selecting a load of commits and dragging-and-dropping them to somewhere else in the history graph. Sadly, I don't know of any GUIs that actually do this. Edit: MJD agrees, and is seeking collaborators for work on such a GUI.

Rebasing occasionally gets stuck when it encounters a conflict. The thing to realise here is that the error messages are really good. Keep calm, read the instructions, then follow them, and everything should turn out OK (but if it doesn't, feel free to skip to the next-but-two GLUE).

Giving up on git mergetool

git mergetool allows you to open a graphical diff/merge tool of your choice for handling conflicts. Don't use it. Git's internal merge has complete knowledge of your code's history, which enables it to do a better job of resolving conflicts than any external merge tool I'm aware of. If you use a third-party tool, expect to waste a lot of time manually resolving "conflicts" that the computer could have handled itself; the internal merge algorithm will handle these just fine, and present you with only the tricky cases.

Learning to use git rebase --interactive

I thought this was going to be hard and scary, but it's actually really easy and pleasant to use, I promise! The name is terrible - while git rebase --interactive can do a normal rebase as part of its intended work, this is usually a bad idea. What it's actually for is rewriting history. Have you made two commits that should really be squashed into one? Git rebase --interactive! Want to re-order some commits? Git rebase --interactive! Delete some entirely? Git rebase --interactive! Split them apart into smaller commits? Git rebase --interactive! The interface to this is extremely friendly by Git CLI standards: specify the commit before the first one you want to rewrite, and Git will open an editor window with a list of the commits-to-rewrite. Follow the instructions. If that's not clear enough, read this post.

This should also be represented in a GUI by dragging-and-dropping, but I don't know of any clients that do this.

Learning about git reflog

Git stores a list of every commit you've had checked out. Look at the output of git reflog after a few actions: this allows you to see what's happened, and is thus valuable for learning. More usefully, it also allows you to undo any history rewriting. Here's the thing: Git doesn't actually rewrite history. It writes new history, and hides the old version (which is eventually garbage-collected after a month). But you can still get the old version back if you have a way to refer to it, and that's what git reflog gives you. Knowing that you can easily undo any mistakes allows you to be a lot bolder in your experiments.

Here's a thing I only recently learned: Git also keeps per-branch reflogs, which can be accessed using git reflog $branchname. This saves you some log-grovelling. Sadly, these only contain local changes: history-rewritings done by other people won't show up.

Reading the code of git-rebase--interactive

Interactive rebase is implemented as a shell script, which on my system lives at /usr/lib/git-core/git-rebase--interactive. It's pretty horrible code, but it's eye-opening to see how the various transformations are implemented at the low-level "plumbing" layer.

I actually did this as part of a (failed) project to implement the Darcs merge algorithm on top of Git. I still think this would be a good idea, if anyone wants to have a go. See also this related project, which AFAICT has some of the same advantages and better asymptotic complexity than the Darcs merge algorithm.

Learning how the recursive merge algorithm works

You don't actually need to know this, but it's IMHO pretty elegant. Here's a good description.

Learning what HEAD actually means

I learned this from reading two great blog posts by Federico Mena Quintero: part 1, part 2. Those posts also cleared up a lot of my confusion about how remotes and remote-tracking branches work.

Learning to read refspec notation

The manual is pretty good here. Once you know how refspec notation works, you'll notice it's used all over and a lot of things will click into place.

Learning what git reset actually does

I'd been using git reset in stereotyped ways since the beginning: for instance, I knew that git reset --hard HEAD meant "throw away all uncommitted changes in my working directory" and git reset --hard [commit hash obtained from reflog] meant "throw away a broken attempt at history-rewriting". But it turns out that this is not the core of reset. Again, MJD has written a great explanation, but here's the tl;dr:

  1. It points the HEAD ref (aka the current branch - remember, branches are just pointers) at a new 'target' commit, if you specified one.
  2. Then it copies the tree of the HEAD commit to the index, unless you said --soft.
  3. Finally, it copies the contents of the index to the working tree, if you said --hard.

Once you understand this, git reset becomes part of your toolkit, something you can apply to new problems. Is a branch pointing to the wrong commit? There's a command that does exactly what you need, and it's git reset. Here's yet another MJD post, in which he explains a nonobvious usage for git reset which makes perfect sense in light of the above. Here's another one: suppose someone has rewritten history in a remote branch. If you do git pull you'll create a merge commit between your idea of the current commit and upstream's idea; if you later git push it you'll have created a messy history and people will be annoyed with you. No problem! git fetch upstream $branch; git checkout $branch; git reset --hard upstream/$branch.

git add -p and friends

Late Entry! The git add -p command, which interactively adds changes to the index (the -p is short for --patch), wasn't much of a surprise to me because I was used to darcs record; however, it seems to be a surprise to many people, so (at Joe Halliwell's suggestion) it deserves a mention here. And only tonight Aaron informed me of the existence of its cousins git reset -p, git commit -p and git checkout -p, which are totally going in my toolkit.


I hope that helped someone! Now, over to those more expert than me - what should be my next enlightenment experience?

pozorvlak: (Default)
Monday, April 20th, 2015 08:29 pm

Calculating the dependency graph of your build ahead-of-time can be fiendishly difficult, or even impossible. Redo brings two, I think brilliant, insights to bear on this problem.
  1. If you have to build the target in the course of calculating its dependencies, that's totally OK, because that's what you really wanted to do in the first place.
  2. You don't actually have to know the entire build graph when you start building; you only need to know enough dependencies such that
    • if a given target T needs to be rebuilt, at least one of the dependencies you know about for T will have been affected;
    • in the course of building T, you will discover the remaining dependencies of T and rebuild any stale ones.
Let's try to write do-files for building OCaml modules.

In default.cmi.do:
redo-ifchange $2.mli
ocamlc $2.mli

In default.cmo.do:
redo-ifchange $2.cmi $2.ml
redo-ifchange `ocamldep $2.mli $2.ml`
ocamlc $2.ml
Note: these do-files will not actually work, because redo insists that you write your output to a temporary file called $3 so it can atomically rename the newly-built file into place, and ocamlc is equally insistent that it knows better than you what its output files should be called. However, this annoying interaction of their limitations is irrelevant to the dependency-checking algorithm, so I'll pretend that they do work :-) I'll try to construct a workaround and post it on GitHub. Update: I have now done so!

The redo-ifchange command says "if you know how to build my arguments, then build them; if any of them changes in the future, then the target built by this script will be out-of-date". So to build X.cmi, we observe that it depends on X.mli (which probably won't need building), and then build it. To build X.cmo, we observe that it depends on both X.ml and X.cmi (which will be rebuilt if need be). Then we invoke ocamldep to get a list of other files imported by X.ml, build those if any are out of date, and finally invoke ocamlc on X.ml to produce X.cmo.

Let's see how this plays out in the following scenario:
  1. We build X.cmo.
  2. We try to build it again.
  3. We edit X.ml, adding a new dependency Y.
  4. We rebuild X.cmo.
First, redo runs default.cmo.do, discovers that X.cmo depends on X.ml and X.cmi, recursively builds X.cmi (determining that it depends on X.mli), and finally compiles X.ml, producing X.cmo.

When we run redo-ifchange X.cmo again, redo will check its database of dependencies and observe that X.cmo depends transitively on X.cmi, X.ml and X.mli but that none of them have changed; hence, it will correctly do nothing.

Then we add the dependency, and run redo-ifchange X.cmo. Redo will again check its database of dependencies and note that X.ml has changed, so it must re-run default.cmo.do. First it notes that X.cmo depends on X.cmi and X.ml: it checks its database and sees that X.cmi depends on X.mli, which hasn't changed, so it leaves X.cmi alone. Next it re-runs ocamldep X.mli X.ml and hands the output to redo-ifchange: this tells redo that X.cmo now depends on Y.cmi. Y.cmi doesn't exist yet, so it builds it using the rules in default.cmi.do. Finally it compiles X.ml into X.cmo.

This system should work provided that all your dependencies live within the filesystem, or can be brought within it; however, if this is not the case then you probably have bigger problems :-)
pozorvlak: (Default)
Saturday, April 4th, 2015 03:38 pm
Learning about compilers made me a more confident programmer - a part of my toolchain that had felt like magic was revealed to be merely a complex (and, sometimes, beautiful) collection of mechanisms, and I gained a much greater understanding of how the code I typed was put into practice by the computer. It occurred to me recently that though I've been using relational databases for years (I first learned about normal forms in 2000ish, from a book called MS Access Unlocked or something similarly unimpressive), I don't actually have much idea of how they work. You enter a SQL query and it gets parsed as normal and then, er... something something query analyser... something B+-tree index query plan AND AS IF BY MAGIC it becomes a nice fast bit of looping and pointer arithmetic.

Clearly this was not good enough. So I asked on Twitter for reading recommendations. Here's what I got.
  • David Meier, The Theory of Relational Databases, 1983 (PDFs). As the name suggests, this looks heavy on the relational algebra and light on implementation.
  • C. J. Date, An Introduction to Database Systems (Amazon link to paper book), 2003. Apparently this "contains a lot about internals. The writing style is quite verbose though."
  • Abiteboul, Hull and Vianu, Foundations of Databases (PDFs). This appears to cover SQL and relational algebra in the first half of the book, and Datalog in the second. Which sounds very interesting, but not quite what I was looking for.
  • Stratis Viglas, Advanced Databases (PDF). Slides from a 2015 undergraduate course given at the University of Edinburgh. Covers topics like on-disk layout, external sorting, query optimization, transaction processing, B+-trees and hash joins - the stuff I was after, in other words.
  • Raghu Ramakrishnan and Johannes Gehrke, Database Management Systems, 2002 (Amazon link, though the first Google hit is a presumably-illegal PDF of the full text!) The course textbook for Viglas' course, this appears to cover relational algebra, practical SQL programming, the DB implementation stuff in the course, and quite a lot more.
  • Julia Evans (aka b0rk) wrote a nice sequence of blog posts in which she delves into SQLite internals; I should re-read these.
  • The SQLite documentation looks pretty good, and includes some information about internals.

It'll clearly take me a while to get through all that! My plan is to read through Viglas' notes, have a go at some of the exercises from his course (which also appear to be online), and then take a look at either Meier's book or AHV. Anything else I should be looking at? Does my strategy sound reasonable?
pozorvlak: (Default)
Wednesday, January 7th, 2015 10:51 pm
Tools I have used today:
  • screwdriver
  • hammer
  • knife
  • epoxy resin
  • Aeropress
  • computer
  • pen
  • scissors
  • pliers
  • soldering iron
  • multimeter
  • Helping Hands
  • siphon
  • superglue
Sadly I didn't have time to use my shiny! new! sewing machine! for the planned trouser-shortening*, but still, this has been a Good Day. And now the demijohn should be sterilised and the siphon flushed through with non-bleachy water, so it's time to rack the hawthorn wine.

* I have two pairs of trousers, with the same nominal measurements, from the same manufacturer, bought a few months apart, and one is a centimetre longer than the other - WTF?
pozorvlak: (polar bear)
Monday, January 5th, 2015 11:21 pm

This summer Andy and I headed back out to the Alps; this time we went to the Chamonix valley, home of Mont Blanc and widely acknowledged as the death-sport capital of the world. We were unlucky with conditions - we arrived right after a week of rain, which meant all the high slopes were loaded and avalanchey, and then we lost several more days to bad weather mid-trip - and we generally had a less successful time than last year. In the end, we bagged one 3700m summit, did an easy (but very enjoyable) rock/scrambling route on the Aiguilles Rouges, went hiking a few times and retreated off two more mountaineering routes. Nonetheless, I feel like I learned a lot; here's my belated attempt to set some lessons-learned down in writing.

Double-check your packing

Andy and I had both moved house shortly before the trip, and in the post-move chaos a few things got left behind. You don't want to spend the first day of your holiday dropping 70EUR on a new pair of softshell trousers, or discovering on the first route that you left the new bottle of suncream at home and brought the nearly-empty one from last year instead. Similarly, I couldn't find my less-tight pair of rock shoes before departure; my tight bouldering shoes got really painful after three hours on the Arête des Crochues.


I don't know what exactly changed, but fitness was a much bigger problem for me this year. Age, weight gain, my asthma getting worse, longer routes, reduced workout frequency, all of the above? Whatever it was, I often found myself struggling to get enough air (which in turn meant I couldn't drink, which in turn meant I couldn't eat). I've since beefed up my base exercise schedule, and will try to do more sport-specific workouts (hillwalking, trail running, climbing) in time for next year's trip. If I need motivation, I flash back to how dreadful I felt on Pointe Isabelle.


We knew in general terms that we needed to do more research, but didn't know how to operationalise that, which meant we couldn't research effectively. Having lists of routes you want to do is good, but what you really need is if-then planning: IF conditions are X, Y and Z, what routes will be in good nick? IF we're at the Foobar hut, what is there to climb in the area? What are the mixed routes, the ice routes, the snow routes? Which routes face in which directions, and from which directions are the approach routes threatened by avalanche? This is the kind of knowledge that lets you move quickly to handle a change in conditions without losing another day to book-scouring.

Compare as many guidebooks as you can

We screwed up on the Arête de Table du Roc because we got to the top of the ridge and found ourselves confronted with an unexpected brèche. We then wasted a long time arguing about where we were before finally deciding to make a long and rather unpleasant abseil down our route of ascent. As it turned out, the route down from the summit proper would have been much easier and less threatened by rockfall. We'd been unable to work out from the Alpine Club guidebook which bit was the summit, but when we got back to the campsite we discovered that it was completely obvious from the topo in Gaston Rébuffat's 100 Finest Routes. Rébuffat's book is a bit large to carry up the mountain with you, but if we'd read the description more carefully in advance we might not have had that problem.


After another season doing classic winter routes in Scotland, particularly the Liathach and Aonach Eagach traverses, my Alpine ropework was a lot more fluid. I'm seconding last year's recommendation that Scottish-style winter climbing is great training for Alpinism.

The Aiguilles Rouges are great

We did the Arête des Crochues in the Aiguilles Rouges (the sub-3000m massif on the other side of the Chamonix valley from the Mont Blanc massif) near the end of our trip, and had a great time: it reminded me a lot of the Skye Cuillin. In retrospect, we should have done this right at the beginning: it would have been a good way to start acclimatising and get our fast-and-light heads on.

The French-language guidebook we had for the Aiguilles Rouges was funny to our British eyes: it gave a topo so detailed it would have its author thrown out of the SMC, apologised for the lack of pitons on the route, and suggested that we might find it amusing to find the route by following rock polish and crampon scratches. Heaven forfend! Who ever heard of such a thing?

Take a non-climbing guidebook

There's a huge amount to do in and around the Chamonix valley when the weather's too bad for climbing. Hilary Sharp's book Chamonix Mountain Adventures was an excellent investment, taking us on some great (albeit damp) hikes. The dinosaur-tracks hike was particularly good. The Alpine Museum, by contrast, was rather disappointing.

To péage or not to péage?

Most of the French autoroute network consists of toll roads ("péages"). If you're driving all the way across France, the tolls mount up fast. As an experiment, we tried avoiding toll roads on the way back. This added about four hours to our journey, but meant we actually saw some of the beautiful French countryside instead of the unchanging embankments that surround the autoroutes. The free Routes Nationales, like their British A-road counterparts, vary a lot in quality: in northern France they're usually wide, fast, straight and pretty, but in the mountains they're usually narrow, full of hairpin bends, and rather frightening. I'd suggest driving on N-roads between Calais and Dijon, and using the autoroutes for getting between Dijon and Chamonix.

There's lots of climbing in Bulgaria

A group of Bulgarians in our campsite kept giving us the hard sell on the merits of Bulgarian rock and winter climbing. To be fair, it does look good, and it's a much cheaper country to travel in than France. So, Bulgaria is now on our list of possible climbing destinations.

Take a pair of approach shoes

I took a pair of fell-running shoes last year and didn't really use them, so this time I left them behind. This was a huge mistake: they'd have been incredibly useful for hiking and the walk-in/out from the Aiguilles Rouges route. I've never owned a pair of purpose-designed approach shoes, but I'm sure a pair of those would have been even better.

Leave plenty of time for hut walk-ins

Mostly because of our lack of effective research, we often found ourselves deciding on a hut with only just enough time to walk in to it, leading to a rushed walk-in and an arrival in the middle of supper. This is a Bad Thing: you want to arrive early so you can ask the hut guardian about recent ascents and conditions, and so you can scope out the start of the route. We lost a lot of time stumbling around in the dark at the start of our attempt on Pointe Isabelle, and this might have made the difference between success and failure.

The other mistake we made on that route was not leaving enough time for the walk-out: given the conditions we probably made the right decision to go down when we did anyway, but it didn't help our decisionmaking that we had to be back on the other side of the Mer de Glace by a certain time in order to not miss the last train.

Eating on routes

We still haven't found an eating-on-routes system we're really happy with. This time we experimented with having a chalk-bag full of snacks each and grabbing a quick bite every so often; I think this helped. We also discovered Nakd bars, which are great, and which can be cheaply and easily duplicated at home :-) I'm wondering if Soylent would be good for mountaineering: put some in your hydration bladder, sip constantly without the need to stop or get anything out of your sack. I'm also considering Plumpy'Nut, or some home-made approximation, and Mike Prior-Jones introduced me to BiFi Rangers and Carazza mini-pizza-sandwich things, which are tasty, shelf-stable and savoury - sometimes you don't want something too sweet. I've since used these with success on the An Teallach ridge in Scotland.

[More photos here!]

pozorvlak: (Default)
Monday, December 29th, 2014 04:38 pm

I was chatting on Twitter last night about the disaster on Everest earlier this year. Michael Story asked me if "climbers mostly think Everest industry = Gomorrah?", and I found it hard to give an accurate answer in the form of Tweets. This post is that answer. I should first explain that though I've read several books about high-altitude climbing and Everest in particular, I've never been to Nepal and never climbed in the Greater Ranges; I'm very happy to be corrected by people with more direct knowledge.

Before we talk about Everest directly, we should talk about the notion of "good style" in climbing, which climbers sometimes grandly call "ethics". The best style of ascent is something like this:

  • climbing a technically challenging route
  • from the ground up, in a single push, using only what you carry with you
  • using only your hands and feet to gain height, not artificial aids
  • without using prior knowledge of the route
  • without using protective gear ("free-soloing")
  • and leaving no trace of your passage.

In short, you should be self-reliant. An ascent in this style is not always feasible, but the closer you can get to it, the better the style of your ascent. In particular, climbers will usually use ropes and protective gear, because dying can mess up your whole day. However, even here there's a hierarchy: if you place protection and use ropes but don't weight them ("free climbing") then you could, in theory, have free-soloed the route, so this is better style than either falling or using gear to gain height ("aid climbing").

The requirement is that you should climb in as good a style as you can, and that you should climb a route in at least as good a style as the first ascensionists used (it's perfectly OK, for instance, to aid-climb most routes on El Capitan). If you can't do that, goes the reasoning, you shouldn't be there: go and climb a route that's within your capabilities rather than bringing the mountain down to your level. The key texts here are Reinhold Messner's The Murder of the Impossible, Lito Tejada-Flores' Games Climbers Play and Cesare Maestri's Compressor Route on Cerro Torre (assuming a 300m line of bolts in a rockface with a diesel generator hanging off the top can be a "text").

You might object that these rules make no sense, or that if you climb in a way that violates my idea of "good style" then you're not reducing my enjoyment of my own climbs in any way, as long as you keep to the "leave no trace" rule. To which I say: welcome to the stupid world of human ingroup/outgroup dynamics and shibboleths. I subscribe to these rules to the extent that I think they enhance my enjoyment and capture something important about what mountaineering gives me, but it's certainly possible to be overzealous in their application - see almost any UKClimbing.com forum thread.

Now let's compare the ideal ground-up on-sight free solo climb described above to the typical guided ascent of Everest.

  • the route, while physically very difficult, is technically straightforward
  • it's climbed "siege style", by establishing a number of intermediate camps and stocking them with gear; worse, this is done by hired porters
  • clients make liberal use of bottled oxygen (it's hard to imagine a clearer case of "bringing the mountain down to your level")
  • clients pull on ropes fixed for them by Sherpas
  • clients are guided up by guides who have intimate knowledge of the route, and make no mountaineering decisions themselves
  • tents, oxygen bottles, human waste, dead bodies, etc, are all left in situ (though this is improving).

Given all that, it should be clear that the climbing subculture would look down on guided Everest ascents even if they didn't kill Sherpas in such numbers.

[The word "Sherpa" is ambiguous; it means a minority ethnic group that lives in the region of Nepal and Tibet near Everest, but is also used as a shorthand for "high-altitude porter". High-altitude porters are the highly-skilled people who carry loads and fix ropes above Base Camp; on Everest they are mostly, but not exclusively, ethnic Sherpas, who have various genetic adaptations which make them better able to do hard work at high altitude than most people. Ethnic Sherpas often use "Sherpa" as part of their names. From now on, I'll use "high-altitude porter" or "HAP" when referring to the job, and "Sherpa" when referring to the ethnic group.]

Now, let's talk about safe working conditions.

The standard Nepalese route up Everest goes through a region of broken glacier called the Khumbu Icefall. It's an objectively dangerous place: exposed to avalanche from above, full of ice-cliffs ("séracs") that could collapse at any moment, and riven with crevasses. Navigation through the ice-fall is complex, and must be completed quickly to minimise exposure to avalanche and sérac collapse. Many clients will not have the mountaineering experience to navigate the Icefall safely on their own (stories abound of clients putting on crampons for the very first time when they get to Base Camp), so at the beginning of the season a cross-expedition team of HAPs called the "icefall doctors" find a safe-ish route through the Icefall and prepare it for clients, fixing ropes along the route and bridging crevasses with ladders. As Jon Krakauer explains here, Western clients and guides now spend much less time in the Icefall - and consequently are at much lower risk - than twenty years ago: liberal use of bottled oxygen and prophylactic drugs means they need to take fewer trips through the Icefall to acclimatise for the summit. But someone still needs to fix the route through the Icefall, and make the necessary trips through it to stock the upper camps; this falls to the HAPs, who consequently have a fatality rate between 4 and 12 times that of US military personnel in Iraq.

This really gets to the core of why so many climbers find the Everest circus distasteful. Mountaineering is about challenging yourself in a beautiful environment, sure, but it's also about self-reliance and intelligent risk-management. If you're going to pay someone else to make all the decisions, remove all elements of technical challenge, and do the most dangerous bits for you, what's the point? Summits are meaningless, but the process of attaining them is not; guided Everest climbs dumb down that process, and risk people's lives for a goal that has had all the point sucked out of it.

[I'm talking about Everest guiding specifically, not the wider mountain-guiding industry. I've hired guides myself, and had some great days and learned a lot from doing so. But there the dynamic is different, more like hiring an instructor.]

I said above that the Icefall is an "objectively dangerous" place. This is climbing jargon, and it means that the dangers (like avalanche) are largely uncontrollable. There are undoubtedly more things that could be done to increase safety for HAPs (Krakauer's article suggests a few), but the Icefall is never going to be a safe workplace. The only way to manage risk in the face of objective dangers is to ensure you spend as little time in the danger zone as possible. But the entire business model of the Everest guiding businesses works against this. If you're going to take inexperienced clients on a siege-style ascent of the mountain then you need to fix a route through the Icefall and transport kit for all the intermediate camps through it. Someone has to do that. Per Krakauer, HAPs can earn between $2000 and $8000 in an Everest season; starting wage for a guide at Rainier Mountaineering Inc. is $125 per day entry-level or $250 for someone with the extremely prestigious and difficult IFMGA certification (and someone guiding on Everest is likely to be more experienced, and consequently better-paid). I'm not privy to Himex's accounts, but I doubt they're an obscenely profitable concern: mounting large-scale expeditions to countries with little infrastructure is an expensive business (small-scale Alpine-style expeditions, by contrast, can be done remarkably cheaply, even in the Himalayas). Paying Western rates to do the amount of dangerous load-carrying that a siege-style ascent of the standard route requires would vastly increase their costs, quite possibly to the level at which the business model would become unsustainable.

Now, let's talk about sex work.

As I tried to demonstrate above, Everest guiding is inescapably bound up with the existence of poor people who'll accept very low wages (by Western standards) to do an unavoidably dangerous job. I understand there's an extensive literature on the ethics of constrained choices under capitalism: if Sherpas can make a good living (by their standards) doing a ludicrously dangerous job, are their Western clients exploiting them? This, as various people pointed out on Twitter, is somewhat analogous to sex work, which is also a dangerous job which many workers are forced into by lack of better options. In both cases, reducing the choices available by banning the dangerous one seems unlikely to be a win. But I think there's one big difference: working as a HAP on Everest is unavoidably dangerous, and I'm not at all convinced the same is true of sex work. I'm on shaky ground here, but AIUI the dangers of sex work are mostly human in origin, and we can hope to make things safer for sex workers by, for instance, changing the surrounding legislation; no law you can pass will stop a house-sized lump of ice from falling off the mountain and triggering an avalanche.

[In the spirit of increasing available choices, let me put in a plug for Sherpa Adventure Gear, a Sherpa-owned clothing firm who do most of their manufacturing in Nepal, donate a portion of their profits to a scholarship fund for Sherpa children, and make really good kit.]

pozorvlak: (Default)
Thursday, October 30th, 2014 01:17 am

I have been learning Russian for the last twelve days. Here's a video of me trying to speak it - feedback very welcome!

I'm doing this mainly because I'm planning a mountaineering trip to Kyrgyzstan next summer. I've travelled in countries where I don't speak the language before, and it's horrible and difficult and isolating. In particular not being able to read the local script (as was the case for me in Thailand, Laos and Cambodia) makes so many practical things harder. Why Russian and not Kyrgyz? Well, friends who've been to Kyrgyzstan tell me that while almost nobody in Kyrgyzstan speaks English, many people there speak at least a little Russian, and I rate my chances of learning Russian to a useful level as much higher than my chances of learning Kyrgyz - Russian is an Indo-European language, it's related to Czech, which I've studied a little, and finding language-learning resources from where I am is going to be much easier. Benny the Irish Polyglot can learn Hungarian in Colombia, but I think I'll make life a bit easier on myself, at least this time :-)

Two resources have been both helpful and inspiring to me: the Coursera course Learning How to Learn, and the aforementioned Benny's website, Fluent In Three Months. I discovered Benny through an interview the LHTL instructors did with him. The other thing that's been helping with motivation is having a Big Hairy Audacious Goal: next to "do a first ascent in the Greater Ranges", "learn enough Russian to get by" seems like (and is!) a mere stepping-stone. The BHAG is helping with the physical training too, but there my thoughts are more like "Does it hurt now? Good. It's going to hurt a lot more at 5000m, so you'd better get on with it".

Historically, I have sucked at learning languages: given how many years I spent studying French at school, it's embarrassing how much I was having to struggle with basic everyday interactions in French on my recent trips to France, and the less said about my Czech or Japanese the better. In Kyrgyzstan I won't have the option of falling back to English, so it's time to HTFU and overcome my inadequacies. My meta-strategy can be summed up as "do all the stuff that you were too lazy, proud, or suspicious to try in previous language-learning efforts": a lot of the techniques covered in LHTL were things I'd heard about but never tried for language-learning.

In more detail, these are the techniques I plan to use:

  • Actually talking to native speakers. Benny's big on this one, arguing that if you want to speak a language then that's what you have to practice. I plan to use italki.com, a website that lets you find native speakers for Skype chats, but if any Russian-speaking readers of this blog would be willing to put up with my attempts, I'd love to speak to you!
  • Recall: the act of recalling information strengthens the neural connections that encode it. LTHL recommends immediately turning your head away from the book and trying to recall it when you first encounter a new fact; in general, you should try and test yourself constantly on any information you're trying to remember. I didn't do this much at school, reasoning that if you don't know something, what do you gain by trying to remember it - surely it's better to expose yourself to the information over and over. A plausible hypothesis, but apparently Science contradicts it.
  • Spaced repetition. The brain needs time to lay down long-term memories, so it's better to spread X hours of practice over a few weeks than to cram it all into X/24 days. This combines nicely with recall - create flashcards, and test yourself on them over days, increasing the period on things you know better. I've been using the smartphone app Anki to learn the 1000 most common Russian words, though unfortunately the deck I found is Russian-to-English, and point 1 suggests I should be concentrating on English-to-Russian, or (better), unnamed-concepts-to-Russian so I don't have to go via English to find the Russian word. I've also made physical flashcards with some basic Russian phrases - hello, goodbye, I am from Scotland, has that mountain been climbed before, that kind of thing. I've never made flashcards before, and I'm not entirely sure why not - laziness?
  • Eating my frogs first, or in other words, doing hard things first thing. My morning routine is now wake up, open the curtains (which I can do without getting out of bed), sit up (this helps the grogginess to drain from my head), pick up my phone and start doing Anki cards. I then usually follow this up with another short Anki session at lunchtime and more when I have a few minutes spare.
  • Mnemonics: instead of simply repeating vocabulary words over and over, I'm trying to invent mnemonics, and preferably vivid images that stick in the mind better. For instance, the Russian for "start" is "начало", pronounced "nachalo"; "на" means "on", so I imagine a line of sprinters poised at the start line, with a cellist on a platform about to sound the note that will start the race. My favourite mnemonic so far is one that [profile] wormwood_pearl invented: the Russian word "понимать", pronounced "ponimat'", means "understand", so I imagine a picky customer going into the automated pony-wash and asking for a particularly complex horse-laundering procedure, which the assistant understands perfectly. I've generally been having trouble coming up with good mnemonics, though. Partly this is lack of practice, and partly it's because many of the words I've tried to learn so far have been function words like and, because, what, then, which, and so on: it's hard to construct vivid images about such things. My mnemonic for "because" is particularly stretched: the word I'm trying to remember is "потому", "patamu", which has a Tam and a Moo in it, and Moo is the sound cows make, and there's a line from a Half-Man Half-Biscuit song "did you ever wonder how they get triangles from a cow? You need buttermilk and cheese and an equilateral chainsaw", and the mathematical symbol for "because" is three dots arranged in a triangle, so I imagine a bonnetted Scotsman called Tam carving a "because" sign out of a cow with a triangular chainsaw. Which is completely stupid, but actually appears to be working. In writing this, it occurs to me that I could help myself to invent mnemonics by taking advantage of the focused and diffuse modes of thinking. When I'm doing flashcards, I'm focused, but creativity requires the diffuse mode to make connections between apparently disparate concepts (preferably after a period of focus). Next time I encounter a word for which I can't think of a mnemonic, I'll write it down so my brain can continue to work on it subconsiously.
  • Chunks and chunk transfer. An essential part of the learning process is the construction of mental chunks - ideas which "hang together", and which can then be recalled as a single unit (like, say, "compiler", or "grammatical gender", or "integration by parts"). Once you have acquired a chunk in one context, you can more easily transfer this knowledge to acquire chunks in new contexts. Here's a description of how I learned to contact juggle by applying a chunk from skiing - I was later able to apply that chunk to rock climbing, where it's called "deadpointing". I had a great Twitter conversation about how deadpointing applies to all sorts of other physical disciplines - fascinating stuff. Anyway, chunk transfer Is My Friend when it comes to language-learning too. I've been able to apply some chunks from Czech (my favourite so far was when I realised that the черно in чернослив (damson, from Google Translate) meant "black" and was cognate with Czech černá, so the слив must mean something like "plum" and was therefore cognate with "slivovice", the name of a potent moonshine distilled throughout Central Europe. I've also, to my delight, been able to transfer a lot of chunks from Nadsat, the teenage slang used in the novel A Clockwork Orange.
  • Not caring about making mistakes. My difficulties with languages are symptomatic of a deeper problem - I've always found enough subjects easy that I could do well in school simply by relying on the things that I could do without difficulty. This, I think, created a fixed mindset - my identity became bound up with "being clever", so I subconsiously avoided situations that would make me look or feel stupid. Tackling hard problems was allowed, but only within the range of subjects that I was good at. My difficulties with the programming language Haskell probably come under this heading too (thing I ought to be good at, but which made me feel stupid), and juggling got a lot less fun (and I started improving a lot less) once I started thinking "I've been juggling X years, I should be better at this by now" rather than "I'm no good at physical hand/eye coordination, I'm allowed to be bad at this". But languages don't come easy to me, so I have to cultivate a growth mindset - more straightforwardly, I need to internalise the idea that sucking at something is an essential step on the way to being good at that thing.
  • Sleep. Sleep physically shrinks neurons, allowing waste products to be removed. It also plays a key role in the consolidation of long-term memories. So getting a full night's sleep as often as possible is very important to successful learning. [Checks time. Oops.]
  • Exercise: as well as being great for general mental health, exercise is apparently nigh-essential for effective learning. Fortunately, I need to get a lot fitter anyway :-)
pozorvlak: (Default)
Saturday, October 25th, 2014 08:44 pm
Я начал изучать русский язык, потому что я хочу пойти альпинизм в Кыргызстане летом следующего года. Я знаю только несколько слов до сих пор, поэтому я использую Google Translate, чтобы написать это. Я уверен, что это полна ошибок. Я надеюсь, что в ближайшее время я буду говорить достаточно русский, чтобы сделать свои собственные ошибки!
pozorvlak: (Default)
Monday, November 11th, 2013 03:21 pm

We've recently moved house, to a refitted Victorian tenement flat in Leith. We're renting it from a lovely couple from Continental Europe, and this I suspect is the reason for one of the few things that annoy me about the place: that every sink in the flat is fitted with mixer taps. Ordinarily this is merely a mild irritant, but occasionally (as happened this morning), they drive me into a towering rage. Let me explain...

I'd taken out the contents of the food recycling bin, but a foul-smelling brown gunge still coated the insides of the bin itself. I was therefore filling the bin with a mix of bleach and hot water, the latter from the bathroom sink. The sink was too small to fit the bin in, so I was filling a pint cup with hot water from the sink and tipping it into the bin. Fortunately it's quite a small bin. My attention lapsed for a moment, though, and the water overflowed, mildly but painfully scalding my left hand. No problem: I could keep filling the hot water with my right hand, while holding my left hand under the cold tap for as long as it took to cool down. Except, oh, wait, mixer taps. Dammit. So I had to turn off the hot tap, put down the cup, turn on the cold tap, and wait uselessly for however long it took for my hand to stop hurting.

Except I had forgotten about the other problem with mixer taps: hysteresis. When you turn off the hot water in a mixer tap system, you see, you don't reset the tap to a safe state: a slug of hot water remains in the pipe, lying in wait for the unwary. And so when I put my sore hand under the tap and turned on the cold water, I was instead treated to a high-pressure dose of painfully hot water onto the already painful area.

And then a few minutes later, while mentally composing this blog post and muttering curses against the inventors of mixer taps and their descendents, yea, unto the seventh generation, the same thing happened to me again.

In conclusion: fuck mixer taps. Fuck them right in their stupid single non-parallelisable pain-causing water outlets.

This post is dedicated to [personal profile] elmyra, who labours under the misapprehension that mixer taps are not only a superior technology, but so obviously a superior technology that the only possible reason they have not been universally adopted can be ignorance of their existence.

pozorvlak: (Default)
Thursday, October 24th, 2013 09:50 pm

I hate making decisions. And I'm right to do so, as the emerging body of evidence on decision fatigue makes clear. But sometimes you have to make a decision in situations where there's no obviously good choice: sometimes the differences between options are trivial, sometimes the differences are significant but the advantages and disadvantages are finely balanced, and sometimes you just don't have enough information to assess what those advantages and disadvantages are, but need to make a decision anyway so you can move on.

Some people advocate rolling a die or tossing a coin in this situation. These people are clearly less indecisive than me. Some people advocate tossing a coin, and if you catch yourself thinking "dammit, the other option would have let me do X" then you have discovered your hidden underlying preference and can go for that one. These people are also clearly less indecisive than me: I do that every time. Finely-balanced advantages and disadvantages; if there were no opportunity cost, there would be no decision to be made.

However, I have discovered a procedure that allows me to deal with many of these situations, and to do so quickly and with minimal stress. If you can't find a good argument for choosing one option over the others, look for a stupid reason instead. And do so in a consistent and general way, to minimise the mental effort required. The short version of my system is

  1. Pick the red one.
  2. If that doesn't work, pick the one with more cats.
  3. If that doesn't work, pick the one with more dogs.
  4. Give up.

The more detailed version is

  1. Eliminate all choices that are less than maximally red. Red, of course, is the Best Colour.
  2. If more than one choice remains, eliminate all choices which are less than maximally feline. More cats beat fewer cats, fluffy cats beat smooth cats, cute cats beat ugly cats, kittens beat adult cats.
  3. If more than one choice remains, eliminate all choices that are less than maximally canine, subject to the rules above with the obvious formal modification applied.
  4. If more than one choice remains, it is officially Too Hard and you are entitled to give up. Go to the pub and order the beer you haven't had before¹.

This may sound stupid, but inventing this procedure has had a noticeable positive effect on my life. It's quick to run through. It turns angsty and tiring vacillation into either a purely mechanical procedure, or a fun game of inventing reasons why abstract things are red, or catlike, or doglike. It works remarkably often: I can't remember when I last had to invoke Rule 4 (though it has had at least one notable success: see below). Hell, Rule 1 is enough most of the time. And, applied consistently over a long period, it causes my life to fill with things that are red, feline or canine, all of which make me happier.

Q: Which of these two equally cute and fluffy kittens should you pick to take home with you?
A: Mu.

Obviously these advantages are not tied to the specific steps listed above. Feel free to substitute your own favourite colour or animals, or to come up with different steps entirely. But I do recommend inventing a procedure like this one if you also struggle to make decisions.

¹ If there is more than one beer available that you haven't had before, drink them all in left-to-right order.

pozorvlak: (babylon)
Thursday, October 3rd, 2013 09:02 pm

We've just moved house, which means that we're waiting for a new broadband connection to be set up. While we're waiting for reliable Internet we've been making our own entertainment, and by "entertainment" I of course mean "beer". It's our first attempt at homebrewing, so we decided to walk before we tried running and bought a beginner's homebrew kit. Which made me think, as one does, of Ninkasi, the Sumerian goddess of beer and brewing, who every day used to brew beer for the rest of the Sumerian pantheon. Beer was one of the crucial enabling technologies for the early urban civilisations in Egypt and Mesopotamia: as well as being fun to consume, it allows you to live in close proximity to other humans without dying of cholera from contaminated drinking water. Beer brewing tech has moved on a bit since Ninkasi's day, but the essentials of the process remain the same: germinate grains to turn the starches into sugars, soak in water to extract the sugars, drain off the resultant liquid (the "wort"), mix with flavourings, ferment in a large vat, drink, enjoy. And since today is National Poetry Day (with a theme of "water", no less), I thought it might be fun to update the Hymn to Ninkasi. Dating from around 1800BC, it's one of the oldest known recipes for beer. Here's an academic translation, and here's a discussion, and a looser but more poetic translation.

Given birth by the flowing water, tenderly cared for by Ninhursaja! Ninkasi, given birth by the flowing water, tenderly cared for by Ninhursaja!

Having founded your town upon wax, she completed its great walls for you. Ninkasi, having founded your town upon wax, she completed its great walls for you.

Your father is Enki, the lord Nudimmud, and your mother is Ninti, the queen of the abzu. Ninkasi, your father is Enki, the lord Nudimmud, and your mother is Ninti, the queen of the abzu.

It is you who watches the included instructional DVD, and is soothed by the presenter's reassuring Australian accent and self-deprecating humour. Ninkasi, it is you who watches the included instructional DVD, and is soothed by the presenter's reassuring Australian accent and self-deprecating humour.

It is you who puts a cupful of bleach in the plastic bucket, and fills it up with hot water. Ninkasi, it is you who puts a cupful of bleach in the plastic bucket, and fills it up with hot water.

It is you who leaves the bucket to sterilise for half an hour, and then rinses it down carefully in the shower. Ninkasi, it is you who leaves the bucket to sterilise for half an hour, and then rinses it down carefully in the shower.

It is you who carries the bucket through to the kitchen, and places it on the counter. Ninkasi, it is you who carries the bucket through to the kitchen, and places it on the counter.

It is you who peels the backing from the thermometer strip, and sticks it to the outside of the bucket. Ninkasi, it is you who peels the backing from the thermometer strip, and sticks it to the outside of the bucket.

It is you who upends the wort can into a saucepan of hot water so that it may flow more easily. Ninkasi, it is you who upends the wort can into a saucepan of hot water so that it may flow more easily.

It is you who opens the can of hopped wort, and guards it even from the noble cats. Ninkasi, it is you who opens the can of hopped wort, and guards it even from the noble cats.

It is you who dissolves the wort in two litres of boiling water, then adds the 1kg packet of fermentable and non-fermentable sugars and stirs vigorously. Ninkasi, it is you who dissolves the wort in two litres of boiling water, then adds the 1kg packet of fermentable and non-fermentable sugars and stirs vigorously.

It is you who wonders what the hell the non-fermentable sugars are there for anyway, and tries to tell your girlfriend about that Holsten Pils advert with Dennis Leary, and remembers that she's too young to remember it. Ninkasi, it is you who wonders what the hell the non-fermentable sugars are there for anyway, and tries to tell your girlfriend about that Holsten Pils advert with Dennis Leary, and remembers that she's too young to remember it.

It is you who tops up the bucket to twenty litres with cold water, stirs and checks the temperature. Ninkasi, it is you who tops up the bucket to twenty litres with cold water, stirs and checks the temperature.

It is you who adds another couple of litres of cold water to ensure the temperature is within the range 21C-27C. Ninkasi, it is you who adds another couple of litres of cold water to ensure the temperature is within the range 21C-27C.

It is you who takes the packet of yeast, tries to tear it open, realises there is no slit in the packet, and looks frantically for a pair of scissors. Ninkasi, it is you who takes the packet of yeast, tries to tear it open, realises there is no slit in the packet, and looks frantically for a pair of scissors.

It is you who cuts open the packet of yeast with a Swiss Army knife and pours it into the bucket. Ninkasi, it is you who cuts open the packet of yeast with a Swiss Army knife and pours it into the bucket.

It is you who curses when your girlfriend reminds you that you were meant to sprinkle the yeast evenly over the surface of the liquid. Ninkasi, it is you who curses when your girlfriend reminds you that you were meant to sprinkle the yeast evenly over the surface of the liquid.

It is you who accepts all the blame if this whole thing goes wrong. Ninkasi, it is you who accepts all the blame if this whole thing goes wrong.

It is you who slots the stupidly-named Krausen Kollar onto the bucket, and then fits the lid. Ninkasi, it is you who slots the stupidly-named Krausen Kollar onto the bucket, and then fits the lid.

It is you who wonders why this system doesn't use an airlock, Googles to find out, and emerges no wiser. Ninkasi, it is you who wonders why this system doesn't use an airlock, Googles to find out, and emerges no wiser.

It is you who hopes that you won't end up with partially-fermented beer all over your new kitchen floor. Ninkasi, it is you who hopes that you won't end up with partially-fermented beer all over your new kitchen floor.

It is you who half-fills the hydrometer tube with the diluted wort and drops in the weighted bulb, to determine the beer's original gravity. Ninkasi, it is you who half-fills the hydrometer tube with the diluted wort and drops in the weighted bulb, to determine the beer's original gravity.

It is you who attempts to read the specific gravity at the meniscus, but can't actually see the meniscus because beer is fizzy. Ninkasi, it is you who attempts to read the specific gravity at the meniscus, but can't actually see the meniscus because beer is fizzy.

It is you who decides that accuracy to +/- 0.001 is probably good enough, and writes down your best guess in the Brewer's Log. Ninkasi, it is you who decides that accuracy to +/- 0.001 is probably good enough, and writes down your best guess in the Brewer's Log.

[It is you who has only proceeded up to this point so far, and is copying the remaining steps out of the instruction booklet. Ninkasi, it is you who has only proceeded up to this point so far, and is copying the remaining steps out of the instruction booklet.]

It is you who waits for four days, periodically checking that the temperature is within the range 21-27C, and then draws off another tubeful of liquid and measures the specific gravity with the hydrometer. Ninkasi, it is you who waits for four days, periodically checking that the temperature is within the range 21-27C, and then draws off another tubeful of liquid and measures the specific gravity with the hydrometer.

It is you who checks the specific gravity daily until it has remained the same for 24 hours. Ninkasi, it is you who checks the specific gravity daily until it has remained the same for 24 hours.

It is you who nervously taste-tests the beer, hoping that it has avoided infection or other problems. Ninkasi, it is you who nervously taste-tests the beer, hoping that it has avoided infection or other problems.

It is you who sterilises the supplied heavy PET bottles, ready to receive the beer. Ninkasi, it is you who sterilises the supplied heavy PET bottles, ready to receive the beer.

It is you who fits the bottling valve to the tube, and opens the tap. Ninkasi, it is you who fits the bottling valve to the tube, and opens the tap.

It is you who fills the bottles with the beer. Ninkasi, it is you who fills the bottles with the beer.

It is you who puts one sugar tablet into each 500mL bottle so that the beer may undergo a second fermentation, then caps them and inverts each one several times. Ninkasi, it is you who puts one sugar tablet into each 500mL bottle so that the beer may undergo a second fermentation, then caps them and inverts each one several times.

It is you who wonders what was going on in that bit in 1984 where the old guy complained that half a litre of beer wasn't enough, seriously, you can barely tell the difference between half a litre and a pint. Ninkasi, it is you who wonders what was going on in that bit in 1984 where the old guy complained that half a litre of beer wasn't enough, seriously, you can barely tell the difference between half a litre and a pint.

It is you who stores the bottles upright at a temperature above 18C (in Scotland, in October) for two weeks while the beer undergoes carbonation. Ninkasi, it is you who stores the bottles upright at a temperature above 18C (in Scotland, in October) for two weeks while the beer undergoes carbonation.

It is you who invites your friends round to try the finished beer; it is like the onrush of the Tigris and the Euphrates. Ninkasi, it is you who invites your friends round to try the finished beer; it is like the onrush of the Tigris and the Euphrates.


pozorvlak: (Default)
Monday, September 23rd, 2013 12:17 am

I got back from my first climbing trip to the Alps a bit over a month ago. My friend Andy and I spent two-and-a-bit weeks in the Écrins climbing routes graded Facile and Peu Difficile, and generally having a blast. I'd wanted to go to the Alps for years, but I'd heard so much about how hard and scary Alpine climbing was that I consistently failed to get a trip together. This year I finally overcame my fears and disorganisation and made it out there, and I found that low-grade Alpine mountaineering (the stuff I was interested in, in other words) was much easier, more fun and less scary than I'd been led to expect. I wish I'd gone five years ago.

Some of the stuff I read beforehand was useful (in particular, I recommend the BMC's DVD Alpine Essentials, which does a wonderful job of demystifying Alpine mountaineering), but I was still left with some misconceptions and gaps in my knowledge. Here's some of what I wish I'd known back then; the usual disclaimer applies.

There is absolutely no need to go to the Dolomites first

People kept telling me that I should go to the Dolomites and do lots of long rock routes before attempting the high Alps. I'm sure the Dolomites are lovely, but this is nonsense. If, like me, you want to do low-grade mountaineering on high mountains, go and do it. Experience of 10+-pitch rock routes is not necessary; I doubt it's even very useful. Moving together on scrambling terrain in big boots is a different skill.

It's nothing like the books

Climbing memoirs and films concentrate disproportionately on the super-hard routes and the times when Everything Went Wrong. It turns out that there are plenty of easier routes too.

Guidebook times are perfectly achievable

I received differing advice on this. All the books and DVDs stressed the importance of completing routes within guidebook time, and only increasing your grade/altitude/length once you were doing so. However, most of the people I spoke to said that this was an unrealistic aim. One guy even said that 1.5x guidebook time was a more reasonable target, but that I (as a Slow Climber) should allow double. In the event, we did almost all of our routes either within guidebook time or only a few minutes over. On the one exception, the South Ridge of the Aiguille Centrale de Soreiller, we took 4.5h versus 3h, but (a) the time quoted was for the shorter variation of the route, and we did the longer variation, (b) we deliberately decided to pitch the exposed summit ridge rather than moving together, having assessed the glacier below and deciding it would be safe to descend later in the day.

It's not that scary

All the books said "you need a few days to get used to the sheer scale of the Alps". I really didn't find this. Granted, the Écrins are not the tallest part of the Alps, but the exposure levels were at most about double what I'm used to from Scotland, and I felt pretty much at home. In fact, I found fear-management much easier than on the average UK roadside crag. If you can handle the Cuillin Ridge, you'll be fine.

That said, learning the Litany Against Fear is not a bad idea. It actually helps, and as an earworm it beats the hell out of Brown Girl in the Ring.

The days needn't be that long

Similarly with the oft-repeated advice that Alpine days can be really really long. Our longest day was ten hours (although we stopped back at the hut - it would have been more like 14 if we'd descended to the valley the same day); I've done 13-hour winter days in Scotland, and 14-hour days in summer. Or 18-hour days if you count epics. There are obviously plenty of very long routes in the Alps, but you don't have to do them. Pick a shorter route and move fast.

I suspect that most advice to newbie Brit alpinists is aimed at hot-headed wannabe Sheffield hardmen. The exposure's huge and the days are long if you think that 10m of gritstone is a long route. Which reminds me of the time I was climbing Curved Ridge in summer with three friends, moving roped together, and two know-it-alls with Yorkshire accents told us that they'd been climbing for thirty years and what we were doing was "not a recognised rope technique". Hey, how about you (a) fuck off back to Stanage, and (b) pick up a fucking book? I'm sure if you ask nicely in the climbing shop they'll help you with the big words.

You'll spend a lot of time downclimbing

The voies normales are, almost by definition, the easiest routes up the mountains. Hence, if there were an easy route off the top, you'd have climbed that instead. You may be able to abseil some sections, but there's no guarantee of this.

It's surprisingly warm up there

My previous experience of climbing in snow and ice had all been in Scottish winter conditions, where your fingers are usually painfully cold, touching the rock ungloved will chill you to the bone, hydration tubes freeze solid, and if you stop for more than a minute you'll need to layer up or dance about or both to keep warm. This was not the case in the Écrins. I did most routes in just a base layer, with my thin belay jacket coming out for summit stops or the occasional fixed belay in the wind. My hardshell only got used during rainstorms in the valley, and my outer gloves were entirely unused. Softshells were more useful: in particular, my Rab Sawtooth softshell trousers were excellent.

On the Barre des Écrins, a guide asked me "do you get many days like this on Ben Nevis?" "Oh yeah, we get some sunny days, even in winter." "No, I meant with the wind!" "Oh, right. In Scotland, we don't consider it windy if you can hold a conversation."

Staying hydrated is hard

Non-freezing hydration tubes make it easier to take a drink without stopping - and you will have very few stops if you're doing it right - but we kept running out of water in the heat. The lack of stops also means you can't do much to adjust your layering system if you get too hot. On our first route - which took a mere 4.5 hours hut-to-hut - I drank the whole of my 2L hydration bladder, then knocked back a 1.5L bottle of water on my own back at the hut.

We also struggled to eat enough on the routes; we never properly hit the wall, but we were definitely suffering from depleted blood-sugar on several occasions. My normal strategy is to scoff chocolate biscuits and sandwiches on belays, or eat while hiking, but this doesn't work when you're moving together on class 3-4 terrain, need your hands to make progress and can't spare the time to stop. I suggested filling our drinking bladders with Gatorade or something similar to Andy, but apparently when he tried that on a previous trip he lost a tooth. Suggestions?

Lassitude is a real thing

I was astonished how little energy I had down in the valleys. The heat sapped the power to do anything except lie about and drink tea.

You'll do a lot of traversing

It turns out that

  1. you have muscles in the side of your calves
  2. they're used a lot when you traverse steep slopes
  3. almost nothing else trains them.


You'll need to switch very quickly between belayed climbing and moving together

File this one under "try not to stop for any reason" - you quite often reach a spot where you can belay the leader over a tricky bit, but the second wants to move off immediately once the rope comes tight. This argues for the use of direct belays off spikes, a technique which horrified me when I first saw it but to which I quickly became accustomed.

Fitness is useful, but you don't have to be an elite super-athlete

I had an ambitious training plan, involving half-marathons and marathons and mountain marathons, but due to various injuries and illnesses and my local gym closing down and other such excuses, I utterly failed to go through with it. Consequently I headed out to the Alps well below my usual level of fitness and carrying about 15kg of excess weight. About a week before I went out, I ran 10km and got delayed onset muscle soreness, so long had it been since I'd done any running. And, you know what? I was mostly fine. The walk-ins to the huts were hard, largely because we were doing them in the heat of the afternoon (see above, "lassitude is a real thing"), and I was pretty spaced out with tiredness on the descent from the Barre des Écrins, but I managed. More fitness would definitely have helped, sure, but lack of fitness wasn't (usually) the limiting factor.

Alpine star fields are amazing

Install a star-map app on your phone before you go. Trust me on this.

So what would be a really useful training plan for that sort of trip? I suggest the following:

  • Do as much hillwalking as you can. If it involves some scrambling, all the better. Practice traversing steep slopes.
  • Practice climbing easy routes in big boots.
  • Practice downclimbing easy routes in big boots.
  • Practice climbing with a full bladder (once again, you don't want to stop if you can avoid it).
  • Do lots of long, grade I/II Scottish winter routes: the kind of thing where you want to move together. This was the only part of this training programme that I actually did, and I'm extremely grateful for it.
  • Practice your French (or the language of whatever country you're visiting). High school was a long time ago for me, and it's embarrassing asking "Parlez-vous Anglais?" all the time. Also, the English-language guidebooks are selective and concentrate a lot on the more aspirational routes; reading the local guidebooks will give you more options.

tl;dr Alpine climbing is the Best Thing Ever. All the fun of Scottish winter climbing without the hot aches.

Me on the summit of La Grande Ruine
Me on my first Alpine summit, La Grande Ruine 3765m. More photos here.

pozorvlak: (Default)
Saturday, September 7th, 2013 11:29 pm
Ah've drank
the specials
that wur in
the fridge

n thit
ye wur prably
haudin back
fer the party

They were great
that strang
that cauld
pozorvlak: (Default)
Monday, August 19th, 2013 09:10 pm
If I ever design a first course in compilers, I'll do it backwards: we'll start with code generation, move on to static analysis, then parse a token stream and finally lex source text. As we go along, students will be required to implement a small compiler for a simple language, using the techniques they've just learned. The (excellent) Stanford/Coursera compilers course does something similar, but they proceed in the opposite direction, following the dataflow in the final compiler: first they cover lexing, then parsing, then syntax analysis, then codegen. The first Edinburgh compilers course follows roughly the same plan of lectures, and I expect many other universities' courses do too.

I think a backwards course would work better for two reasons:
  1. Halfway through the Stanford course, you have a program that can convert source text into an intermediate representation with which you can't do very much. Halfway through the backwards course, you'd have a compiler for an unfriendly source language: you could write programs directly in whatever level of IR you'd got to (I'm assuming a sensible implementation language that doesn't make entering data literals too hard), and compile them using code you'd written to native code. I think that would be pretty motivating.
  2. When I did the Stanford course, all the really good learning experiences were in the back end. Writing a Yacc parser was a fiddly but largely straightforward experience; writing the code generator taught me loads about how your HLL code is actually executed and how runtime systems work. I also learned some less obvious things like the value of formal language specifications¹. Most CS students won't grow up to be compiler hackers, but they will find it invaluable to have a good idea of what production compilers do to their HLL code; it'll be much more useful than knowing about all the different types of parsing techniques, anyway². Students will drop out halfway through the course, and even those who make it all the way through will be tired and stressed by the end and will thus take in less than they did at the beginning: this argues for front-loading the important material.
What am I missing?

¹ I learned this the hard way. I largely ignored the formal part of the spec when writing my code generator, relying instead on the informal description; then towards the end of the allocated period I took a closer look at it and realised that it provided simple answers to all the thorny issues I'd been struggling with.
² The answer to "how should I write this parser?" in an industrial context is usually either "with a parser generator" or "recursive descent". LALR parsers such as those produced by Yacc are a pain to debug if you don't understand the underlying theory, true, but that's IMHO an argument for using a parser generator based on some other algorithm, most of which are more forgiving.
pozorvlak: (Default)
Sunday, June 2nd, 2013 09:15 pm
I've been learning about the NoSQL database CouchDB, mainly from the Definitive Guide, but also from the Coursera Introduction to Data Science course and through an informative chat with [personal profile] necaris, who has used it extensively at Esplorio. The current draft of the Definitive Guide is rather out-of-date and has several long-open pull requests on GitHub, which doesn't exactly inspire confidence, but CouchDB itself appears to be actively maintained. I have yet to use CouchDB in anger, but here's what I've learned so far:

  • CouchDB is, at its core, an HTTP server providing append-only access to B-trees of versioned JSON objects via a RESTful interface. Say what now? Well, you store your data as JavaScript-like objects (which allow you to nest arrays and hash tables freely); each object is indexed by a key; you access existing objects and insert new ones using the standard HTTP GET, PUT and DELETE methods, specifying and receiving data in JavaScript Object Notation; you can't update objects, only replace them with new objects with the same key and a higher version number; and it's cheap to request all the objects with keys in a given range.

  • The JSON is not by default required to conform to any particular schema, but you can add validation functions to be called every time data is added to the database. These will reject improperly-formed data.

  • CouchDB is at pains to be RESTful, to emit proper cache-invalidation data, and so on, and this is key to scaling it out: put a contiguous subset of (a consistent hash of) the keyspace on each machine, and build a tree of reverse HTTP proxies (possibly caching ones) in front of your database cluster.

  • CouchDB's killer feature is probably master-to-master replication: if you want to do DB operations on a machine that's sometimes disconnected from the rest of the cluster (a mobile device, say), then you can do so, and sync changes up and down when you reconnect. Conflicts are flagged but not resolved by default; you can resolve them manually or automatically by recording a new version of the conflicted object. Replication is also used for load-balancing, failover and scaling out: you can maintain one or more machines that constantly replicate the master server for a section of keyspace, and you can replicate only a subset of keyspace onto a new database when you need to expand.

  • CouchDB doesn't guarantee to preserve all the history of an object, and in particular replications only seem to send the most recent version; I think this precludes Git-style three-way merge from the conflicting versions' most recent common ancestor (and forget about Darcs-style full-history merging!).

  • The cluster-management story isn't as good as for some other systems, but there are a couple of PaaS offerings.

  • Queries/views and non-primary indexes are both handled using map/reduce. If you want to index on something other than the primary key - posts by date, say - then you write a map query which emits (date, post) pairs. These are put into another B-tree, which is stored on disk; clever things are done to mark subtrees invalid as new data comes in, and changes to the query result or index are calculated lazily. Since indices are stored as B-trees, it's cheap to get all the objects within a given range of secondary keys: all posts in February, for instance.

  • CouchDB's reduce functions are crippled: attempting to calculate anything that isn't a scalar or a fixed-size object is considered Bad Form, and may cause your machine(s) to thrash. AFAICT you can't reduce results from different machines by this mechanism: CouchDB Lounge requires you to write extra merge functions in Twisted Python.

  • Map, reduce and validation functions (and various others, see below) are by default written in JavaScript. But CouchDB invokes an external interpreter for them, so it's easy to extend CouchDB with a new query server. Several such have been written, and it's now possible to write your functions in many different languages.

  • There's a very limited SQL view engine, but AFAICT nothing like Hive or Pig that can take a complex query and compile it down into a number of chained map/reduce jobs. The aforementioned restrictions on reduce functions mean that the strategy I've been taught for expressing joins as map/reduce jobs won't work; I don't know if this limitation is fundamental. But it's IME pretty rare to require general joins in applications: usually you want to do some filtering or summarisation on at least one side.

  • CouchDB can't quite make up its mind whether it wants to be a database or a Web application framework. It comes by default with an administration web app called Futon; you can also use it to store and execute code for rendering objects as HTML, Atom, etc. Such code (along with views, validations etc) is stored in special JSON objects called "design documents": best practice is apparently to have one design document for each application that needs to access the underlying data. Since design documents are ordinary JSON objects, they are propagated between nodes by replications.

  • However, various standard webapp-framework bits are missing, notably URL routing. But hey, you can always use mod_rewrite...

  • There's a tool called Erica (and an older one called CouchApp) which allows you to sync design documents with more conventional source-code directories in your filesystem.

  • CouchDB is written in Erlang, and the functional-programming influence shows up in other places: most types of user-defined function are required to be free of side-effects, for instance. Then there's the aforementioned uses of lazy evaluation and the append-only nature of the system as a whole. You can extend it with your own Erlang code or embed it into an Erlang application, bypassing the need for HTTP requests.

tl;dr if you've ever thought "data modelling and synchronisation are hard, let's just stick a load of JSON files in Git" (as I have, on several occasions), then CouchDB is probably a good fit to your needs. Especially if your analytics needs aren't too complicated.