This Week in D April 17, 2016

Welcome to This Week in D! Each week, we'll summarize what's been going on in the D community and write brief advice columns to help you get the most out of the D Programming Language.

The D Programming Language is a general purpose programming language that offers modern convenience, modeling power, and native efficiency with a familiar C-style syntax.

This Week in D has an RSS feed.

This Week in D is edited by Adam D. Ruppe. Contact me with any questions, comments, or contributions.

Statistics

In the community

Community announcements

See more at the announce forum.

Editorial

A major thread on the forum this week had Andrei Alexandrescu saying "inout must go". He argues that it is too much complexity for too little gain.

My opinion on inout is actually that it is worth it, though it could use some tweaks. Here, I'd like to explain the reasoning I use and go over some D features that I don't think are worth the hassle. Controversy! But, remember, like most the things in this publication, it is my opinion and I do not have say over the development of the D language.

inout does target a fairly small problem that can also be attacked with templates, indeed, but it also has a fairly small impact on code. It is used where you use it and has little impact on the rest of the program. Andrei did find a legitimate intersection with generated code: a template, like isInputRange that takes an inout(T)[] (let's call it R) could potentially declare a local variable of type R, which fails because one of the inout rules precludes declaring an inout local in a function without inout params.

That does suck. I'm not sure that rule carries its weight, exactly because this situation invades other code, though I actually believe isInputRange is poorly implemented anyway. (I would love to see isInputRange implemented in terms of a checkInputRange, which is implemented with reflection primitives and returns an error report, but I digress.) But, the rest of inout is limited in scope to where it helps.

A specialized tool that only helps in a specific situation always looks useless when you are outside that situation, but as long as it doesn't hinder you outside, the cost is fairly limited. Yes, some new reader might not know what it is upon first look, but they can learn. It is expected that a domain master will need to learn special tools to solve their problem, even when general purpose tools get the job done.

When my truck's power steering failed last year, I set out to fix it with my general purpose tools. My wrench could remove the failed pump, though it took me a while. I improvised a pulley remover with a couple wrenches and a hammer. It worked, but it took me a while. When it was time to put everything back together, I couldn't get it done with my tools and purchased a cheap pusher for the one-time problem I saw myself facing. You might see the pattern now: it worked, but it took a while... almost ten hours of me trying to do this myself with my tools.

Granted, I'm not experienced in motor vehicle repair either, but there was an enormous contrast when a friend of a friend who works in a garage showed me how he'd do the same repair: with a long socket set, he removed the pump in about two minutes, rather than the twenty I spent, and never banged his fingers under the hood. Then, using a professional pulley puller with a power driver (I don't recall exactly what the driver was), he replaced the component in a matter of minutes - the same thing that took me hours with my tools.

The mechanic's professional tools cost hundreds of dollars, need to be used properly, and aren't terribly useful outside of working on cars. But, they save a LOT of time when you are in that situation they are designed for. The time and money the garage invested in these tools will be plenty worth it when they use it, able to fix a lot more cars - and make a lot more money - in the same time as their competitors without the proper tools.

D tries to go a level deeper: it isn't just a collection of professional-grade tools, it is a collection of facilities to create professional-grade tools. Its metaprogramming and modeling capabilities do not necessarily solve any problem directly, but you can use these capabilities to make more facilities to solve almost any problem that you come across.

This view is what cements D's claim to be a general purpose systems programming language.

But, at the same time, D seeks to be practical and convenient. Being able to make specialized tools for specialized tasks gives D power, but if you have to make specialized tools for every task, it gets obnoxious. Common tasks have common solutions. In D, you can build classes out of structs and other pieces (and I think you should actually know how! it is useful to know the innards of any feature so you can reorganize the constituent blocks when the need arises), but they are also baked directly into the language with the class keyword.

Such "just works" features for common tasks are what anchor D as a practical and convenient modern language. But, if it is trivial to construct and to use already, is there a need for one? Oh, maybe, but probably not.

Let's summarize my meandering thoughts into some bullet points to consider:

Now, let's get back to inout and apply these points:

Is there a general purpose option? Yes: templates can do the same thing, as Andrei illustrated in the thread. Is it still easy to use? Well, to really get the same result, the templates must be constrained, the three options must be written out manually anyway in an interface, and overridden separately in child classes... so, there are important places where the answer is no.

Does inout's existence affect code that isn't interested in it? Currently, yes, though it is fairly limited even now, and a tweak to the rules can change that. How does it intersect with other features? Well, honestly, I don't know for sure, but the fact that I can write 100,000 lines of code barely thinking about inout outside the functions that use it is a really good sign.

Finally, well, the question we should probably ask first: how much of a help is it? I think it is pretty useful, frankly, I think it is const that works.

const in D is meant to bridge the gap between mutable and immutable - a function that can take either. But you cannot return that view. Sure, you can take mutable or immutable, but that's all you can do: take it. You cannot do much else.

Suppose you want to do some kind of property getter function:

class Item {
	Item parent;
}

Here, parent is readable and writable to the public. You could make an item const to make it nonwritable, but then you cannot call any mutation methods on it either. The simple solution is to do a property:

class Item {
	private Item _parent;
	Item parent() { return _parent; }
}

Great! But now, if you have a const item, you cannot call parent. If you make it const, you cannot call mutable methods on the parent, even if the item itself is mutable.

The answer? Some kind of mutability wildcard.... inout does the job perfectly.

class Item {
	private Item _parent;
	inout(Item) parent() inout { return _parent; }
}

That works flawlessly - so flawlessly, that I have a hard time justifying ever writing const methods when inout is available. (const function parameters are a different story, simply because there might be two parameters where only one is returned; only one parameter is the this, if you will, and inout makes the most sense on the this. Other params can be plain const, seen, but not changed.)

A template here is clunky, but works (at least until inheritance comes in, but you can forward, so clunky.)

I think inout is worth it when you want it. I want to keep it.

So, I'm pro-inout, but I promised to talk about features that I don't feel are worth it. I hinted at a list in the forum thread, including: nothrow, @nogc, @safe, and pure.

I actually think pure is brilliant; on paper, I love it. But, in practice... I very rarely use it. And I often don't care, if you never use pure, you don't worry that functions aren't marked pure... until someone complains that I didn't use it, and they want to but can't because it is a viral attribute.

This is one of the only things that make me want to say "D sux" as I write it:

pure @safe nothrow @nogc // this line sucks!
int add(int a, int b) { return a+b; } // this line is OK

When writing this function, I don't care about any of those attributes. I don't need the compiler to verify those things, the function is simple enough that I can eyeball it easily. (This is an unrealistically simple function, but there are quite a lot of real functions like this too: my color.d has a lot of trivial functions, complicated by all the attributes added just for OTHER users.)

When writing a complex function, these attributes may be useful, but they spread. Massive failure on bullet #3 of my list: it is very intrusive on code that just doesn't care.

Changing the defaults could change this calculus, sometimes. Is pure a case of this? If it were the default, undecorated functions would not be allowed to access global variables nor call functions marked impure.

if you make a function impure, it can do whatever it wants; it releases this individual function from the requirements. You do that because you have some specific reason, in your own implementation, to do it. That reason might be calling another impure function or accessing a global variable, but it is something you or someone else deliberately wrote - and can reconsider.

It isn't failing just because some other programmer didn't add a totally irrelevant item to her or his function (which may not have even existed in the language at the time the function was written!). It is failing because of some conscious decision.

Problem is: it would still kinda suck that it would bubble ALL the way to main if you ever used a single impure function, anywhere in the program. (to be honest, I think while pure by default makes it far more useful, an exception should actually be made for main, which would logically be impure by default as the root of any call tree - virtually ANY program is going to have an impure main to do I/O and writing it every time feels silly. main is a special function anyway, so I don't feel bad putting special rules on it against other defaults.)

But, while there'd be a trunk all the way up of impure, huge swaths of a program are still pure, and using it by default gives you the benefits it promises without affecting most code that just doesn't care.

pure in its current form, while beautiful in so many ways, sucks because I have to write it when I don't care just in case someone, somewhere, at some time, might care.

@safe is in a very similar boat: as it sits now, you have to write it all over, even when you just don't care. It affects all kinds of things outside the field it aims to help - failing my definition of a good specialized tool. But, if it were the default, the tables would turn, and thanks to @trusted, you can stop the bubbling up at any time (and if written correctly, you maintain the promise of @safe)!

I currently don't care about @safe - I tend to write D like C or C++ in this area - but sometimes I HAVE to list the annoying attributes for the future. If it was the default, I'd just add @trusted somewhere and actually pay attention it. My code would probably be better... and probably shorter.

I'd be pro-@safe, iff it is by default. It sux now, but doesn't have to suck.

nothrow... meh. I don't like writing nothrow on functions that I write, because I don't care, but the good thing is nothrow isn't as viral as pure: yes, it has a @trusted analog: try {} catch(Exception e) {}! Yes, just wrap it and catch any exceptions and you can still use nothrow with my functions.

So yeah, nothrow is meh but I don't hate it enough to remove it. I most certainly do NOT support making it default though: exceptions are a REALLY useful thing and I use them basically everywhere. If I had to write throws everywhere (and make no mistake, that'd be viral all the way up to main), I'd fork the language.

@nogc, oh, @nogc. I hate you. My view is that it is a worthless attribute that gets its viral tentacles all over unrelated code for no real benefit, and I think it was added purely for marketing purposes - without even thinking it through, much less actually doing the extreme tedium of making it work by marking every function that exists and might possibly be valid with the attribute.

Inference on templates lightens the pain... slightly. Very slightly, it still sucks, even in template heavy code.

Mind you, writing D that avoids the GC is easy, but writing D that uses @nogc is hard. It is very easy to verify no GC calls with runtime tests, which you should be doing anyway with performance sensitive code - test first before doing intrusive optimizations. (BTW, easy optimizations are easy, just do them. But hard optimizations should be focused where you need them, and determining where you need them is a job for runtime profiling.)

@nogc, on the other hand, has a lot of false positives. Any little function someone writes but doesn't aggressively annotate will trigger a compile failure - and you can't work around it with things like @trusted or catch. Great library doing great work, but its author didn't care for writing the attribute quad for no immediate gain? Sorry, you are locked out unless you kill @nogc from your whole call chain now.

What if it was the default? Changing defaults would save pure and @safe, so what about @nogc? Nope, now the typical program is going to be saying @gc EVERYWHERE, all the way up all the call chains for any time you use any of the allocating functions - even if they only do on a very rare branch that you don't care about in your performance profile. So much for practical and convenient.

Besides, both @nogc and a hypothetical @gc unfairly singles out the built-in garbage collector as a scary thing. I'm not going to pretend that D's garbage collector is a great implementation. It isn't. But it is good enough for a LOT of things and far from the only pitfall in a performance-sensitive program.

What about computational complexity? Or just a plain old slow function? Or blocking I/O? Or some library allocator with similar pitfalls to the built-in GC? The list goes on and on: writing a well-performing function under your circumstances may be trivial or it may be hard. In neither case is @nogc likely to be a panacea, or even much of a help relative to runtime profiling.

@nogc also fails on the intersection angle: other language features just don't jive well with its strict static interpretation. Yes, again, using D without the GC isn't really hard. If you want to do that with your program, you easily can... but you do need to know some kind of allocation scheme for a great many tasks.

If you throw an exception without the GC, easy, just remember to free it when you catch it. You can do array appends with library containers. You can make your own delegates with functors and the such. But, all of these aren't solved by "I will not use the GC", but rather by "I will use this technique instead". This *specific* technique instead: your catch blocks will know to call free. Your delegate-receiving functions will promise to just borrow, or else be changed to take a self-contained object.

You need an active solution to memory management somehow. Even if you decide that you just won't allocate memory, you've gotta write the program with that in mind.

@nogc fails on three of my bullets: it affects code that doesn't care, in intersects poorly with other language features, needing an active solution not just a prohibition, and it isn't really of that much help in performance.

But, what about the general purpose tool maker? This might be what irks me the most: instead of @nogc, I'd actually prefer to have our own attribute definition and function checkers, then @nogc could just be a library artifact using that general purpose metaprogramming facility. Then, we could use the same thing for computational complexity, or blocking I/O, or whatever other problem of the week we cross in the future. (I'd probably still say it sucks if people asked me to write a hundred custom attributes on all my leaf functions to account for their higher up trees, but at least there'd be more potential behind it.)

A general purpose @nogc seems possible to me, then it might be more interesting. But still, ugh.

Finally time to wrap this up. My take: inout must stay. It makes const useful. pure and @safe should be the default, it'd make them useful when you don't care, though in the case of pure, impure might end up being the new thing to hate. nothrow is meh, take it or leave it. @nogc must go.

But, my final opinion isn't as important as you applying your own reasoning to things when making feature proposals - to remove or to add. Consider the value of specialized tools for special purposes. Consider the costs of them creeping into other people's code when they won't benefit that programmer. Consider generalization, but never forget practicality and convenience.

Only after weighing ALL of these things, and surely more things too, should we make a real decision. A smart language may trim features, yes, but minimal is not automatically smart.

Learn more about D

To learn more about D and what's happening in D: