This Week in D September 6, 2015

Welcome to This Week in D! Each week, we'll summarize what's been going on in the D community and write brief advice columns to help you get the most out of the D Programming Language.

The D Programming Language is a general purpose programming language that offers modern convenience, modeling power, and native efficiency with a familiar C-style syntax.

This Week in D has an RSS feed.

This Week in D is edited by Adam D. Ruppe. Contact me with any questions, comments, or contributions.

Statistics

New Releases

LDC

LDC 0.16.0 alpha2 is out!

This release brings the 2.067.1 frontend, to ldc and supports LLVM 3.1-3.7 for well-optimized generated code.

DMD

After a quick beta, Release D 2.068.1 is ready!

This release brings no new features, just several bug fixes from the 2.068 release.

In the community

Community announcements

See more at digitalmars.D.announce.

Community interview

This week we have an interview of Atila Neves, again courtesy of Joakim. Huge thanks to both of them! Their interview follows:

Atila Neves is a regular contributor to dub, the D repository for third-party tools and libraries. He has written reggae, a meta-build system; cerealed, a binary serialisation library; and unit-threaded, a multi-threaded unit-testing framework which is being considered for std.experimental in the D standard library, phobos.

Q: You've used a fair amount of programming languages, what stands out for you when writing D?

A: Very little boilerplate needed. And in the rare cases where there is any, I can make the compiler write the code for me. There's nothing I hate more in programming that repeating myself. Otherwise, D is very close to being the language I'd come up with if I invented my own. In fact, with a default of @safe @nothrow pure for functions and immutable for variables it'd be 99.9% there.

It's the little things, like the fact that defining opCmp just does what you expect it to. inout means not having two versions of functions, like what's required in C++. No header files as well, of course, but every language with modules has that.

And ultimately, my big D hammer: compile-time reflection. Examples of compile-time reflection usage are in all of my first D projects, unit-threaded, cerealed and unencumbered. UDAs particularly make customizing extremely easy.

Q: What's the status of reggae and where do you want to take it? Any plans to add features similar to ccache or distcc, ie caching or distributed builds?

A: The status is that I think it's ready to be used, but I'd expect a bug here or there. The only way they'll be weeded out is by being used on real-life builds. Distributed/cached builds are a great idea, but there's a lot more that I want to do first.

My two priorities at the moment are to write a Makefile to reggae converter to make it easy to migrate, and to enable build descriptions to be written in scripting languages. The reason for the latter is that there are many build system options out there, and each of them seems to cater to a niche of developers. Rubyists use Rake, Haskellers Shake, Pythonistas Waf or SCons, and so forth. I want reggae to conquer the world, so I want to make it possible to write build descriptions in Python, Ruby, Lua, Perl, Emacs Lisp even!

Another thing is to work on a killer feature: the fact that reggae can output a binary backend. Theoretically, this could be similar to "only D can do this" compile-time regular expressions: a custom-built binary executable that knows everything about your build and therefore is as fast as it gets. Every time I think there's not much else to implement, I think of another important feature. Replacing CMake is a tall order.

Q: How do you plan on implementing those two priority features for reggae?

A: One of them is done: it's possible to write build descriptions in Python right now. I put the work in to define a generic interface that any language could use and started with Python. It should be easy to add any other scripting language now.

As for converting makefiles, the goal there would be the current D makefiles for phobos, druntime and dmd. I found a Python library for parsing makefiles, I intend to use it to transform the Makefile AST into a reggae description.

Q: Have you thought about automating build configuration, like ekam? I think that's the future, with some manual nudges from the dev.

A: I've seen it before, but from experience I'm sceptical that it can do all the weird things I've had to specify before in C and C++ builds. Such an approach would be a lot more likely to work for D I think, where tests are found easily since they're part of the language.

Having said that, reggae is a fair bit of the way there already: dub configurations are built by specifying them, dub files themselves are mostly fire-and-forget, and the high level rules for C and C++ mostly work by telling them directories, not individual source files. What Ekam can figure out for the main binary would be:

import reggae;
alias objs = objectFiles(Source!(["."]));
alias app = link!(ExeName("app"), objs);
mixin build!app;

Which I don't think is all that much work. You'd still have to add -I flags where appropriate. The other thing that comes to mind with Ekam's approach (tried to find this, tries to find that) is speed. The reason I discovered Ninja was because I was fed up with no-op builds taking 8s on a 200k C++ codebase. I can only imagine what looking for files and what to link would be like on a multi-million SLOC build.

Q: Why cerealed? Orange already exists, why write another serialization library?

A: I'm not sure I knew about the existence of Orange when I started work on cerealed. Even if I did, I would've written it anyway, since one of the goals was to understand how compile-time reflection could make the task easier. Also, the two libraries focus on different things: at least at the time I don't think Orange did binary serialisation, and there was more boilerplate than I was comfortable with. That's what I remember, anyway. Cerealed was built first and foremost to make implementing network protocols easier. I think I've succeeded since I've implemented quite a few now with it. I might extend it to handle JSON as well, I've just not needed it. Or if I do, maybe I'll just use Orange.

Q: You wrote your own multi-threaded unit test framework, unit-threaded, which you're now trying to get into phobos: how was the experience of writing it? Was it much work to integrate with the built-in unit tests and extend them?

A: It was my first D project and it was fun all the way. My brain hurt a few times whilst I got to grip with what ran at compile-time and what ran at run-time, but by the end I was a better programmer for it. In the beginning, integrating the built-in unit tests was a bit frustrating: the best you could do then was run all the unit tests in a module as a single function. But since then __traits(getUnitTests) made it so easy as to make the use of test functions obsolete! I never used test classes myself, so neither of those two are in the Phobos PR, just regular built-in unittest blocks.

Q: Please tell us about yourself: who you are and where you're from, what programming languages you used before D, and take us from your experience first discovering and using D to writing your own D libraries and tools.

A: I was born in Brazil, spent the last third of my childhood in the UK, and have a Portuguese passport as well on account of having lived there for 10 years, but now I live in Switzerland. I have a PhD in Particle Physics but always enjoyed programming more and didn't know which of the subjects to pick when I went to university. In the end I picked Physics, thinking that I could always learn programming by myself if that's what I wanted to do. I was right.

Before I chanced upon D, I learned these languages in chronological order: Basic (the ZX Spectrum dialect), Pascal, C, Common Lisp, Java, C++, Perl, Ruby, Python. And after D, Haskell.

I heard about D in 2007 or 2008, but back then there was the whole Tango vs Phobos debate and I decided to come back after it was resolved. I watched nearly all of the DConf2013 videos when they went online and coincidentally mentioned D to a colleague who had a copy of TDPL and asked if I wanted to borrow it. I said yes, read it from cover to cover on a Sunday afternoon and was instantly hooked. This also happened to be around the time I got disillusioned with C++11, so it all came together to conspire into making me a D programmer.

I started writing D to explore its compile-time reflection capabilities. I'd heard about it from the DConf 2013 videos and didn't really know what it meant, so I tried doing something useful that required it. As it just so happened, I'd recently written two libraries in C++11 that would benefit from said capabilities and wondered how much easier they would be to write in D. A lot, as it turns out. They became unit-threaded and cerealed. After that, a colleague who's a Go fan issued a lunch-time challenge and that became my D implementation of an MQTT broker. These first experiences were all great and that's what led me on the path I am today.

Q: Do you use any D code at work? What do you do for a living?

A: I've written a few tools in D. Lately I even got to rewrite some work features from scratch in D and who knows, that might even get released to customers. Right now I work for Cisco, taking care of a legacy C codebase written by another team that does network performance monitoring.

Q: Can you talk about some concrete features or benefits of D you've run across when writing one of your D libraries/tools?

A: The main feature I've mentioned before: never having to repeat myself. I try to only use mixins when I have to, but when I do the overall complexity goes down globally. The other main benefit is to make template code so easy that it's natural to use it. To give an example, the C++ version of cerealed used traditional OOP dynamic dispatch and so that's how the D version was born. Later on I realised that the direction the (de)serialisation is happening in is never a run-time decision; this is always known at compile-time. So why use dynamic dispatch? I switched everything over to static dispatch with generic code (and, I learned later on, design by introspection) and it got a lot faster. I _could_ have written it similarly in C++, but I didn't because it never occurred to me. And it never occurred to me because C++ templates are painful.

Then there are the little things: final switch, not having to write code to print out your structs and enums, getopt.

Q: What problems with the language or stdlib have you run into?

A: The good thing about D and stdlib problems is you can always submit a PR to fix them! I think the only things I had to write myself was a parallel filter and a parallel map that used closures instead of regular functions. My main problem is attributes. I try to make everything @safe pure @nothrow, but then I call a Phobos function and it all has to change. To make matters worse, I don't think I've caught a bug so far because of them, and they're so much work. Weeks ago I tried using Algebraic and immediately gave up since all of a sudden everything became @system. I don't think I'm still 100% on when and how to use @trusted either.

The other thing that usually gets to me is how many times I have to use auto when I want to make everything immutable. I don't like mutable state and want the bare minimum possible in my programs. Sometimes D (or a library type) makes me use mutable state. I always feel dirty.

Q: What do you love about D? Hate?

A: Love? The philosophy behind the language and its design decisions, what's considered a good idea by the D creators is usually in line with my opinions and preferences. An example would be that users should be able to do as the language designers, and that none of the types in the library are "magical" - you can make one too! That's the kind of thing that turned me away from Go, for instance.

D's philosophy gels with my number one pet peeve in programming: repeating myself. I never have to do that in D. The community, the ability to participate and shape the course of the language and standard library. How productive I am with it. Hate? Nothing.

Learn more about D

To learn more about D and what's happening in D: