This Week in D March 15, 2015

Welcome to This Week in D! Each week, we'll summarize what's been going on in the D community and write brief advice columns to help you get the most out of the D Programming Language.

The D Programming Language is a general purpose programming language that offers modern convenience, modeling power, and native efficiency with a familiar C-style syntax.

This Week in D has an RSS feed.

This Week in D is edited by Adam D. Ruppe. Contact me with any questions, comments, or contributions.

Statistics

The team is preparing for the next official dmd release, and is only merging major bug fixes and regressions found in the beta at this time.

Major Changes

Last beta for D 2.067 released; the release candidate is right around the corner.

In the community

Community announcements

See more at digitalmars.D.announce.

Significant Forum Discussions

See more at forum.dlang.org and keep up with community blogs at Planet D.

Community Interview

We have a new user contribution this week: Joakim interviewed Sönke Ludwig and discussed dub and vibe.d. What follows is the text of that interview, courtesy of Joakim:

Sönke Ludwig is the primary developer behind DUB, the D package manager, and the vibe.d web framework, which was recently chosen to be bundled alongside the reference dmd compiler.

- How did you first come to start the vibe.d project and why? Was it your first D project or were there other attempts before?

- I first heard about D, I think, in a magazine article around maybe 2005 and got all ears immediately. My growing dislike for C++ (lacking an overall better alternative) grew over the years and the cleaned-up syntax and the module system of D alone were huge selling points. However, things were still too unstable back then: platform support was lacking and I didn't feel ready to ditch the concept of "const" (which didn't exist before D 2.x), so that the starting shot for me was when the first 2.000 alpha of DMD was released back in 2007.

My first large D project was the conversion of my C++ game engine and GUI framework to D. Bit for bit, going over the course of multiple years, finally the D version had more features than the old C++ code base - at about half the number of code lines and those were also much shorter on average! This framework also forms the basis of most of my current commercial projects. I'd like to publish parts of it one day, maybe using some kind of dual-licensing scheme, when I have enough time to properly maintain it.

Vibe.d was born later, in 2012, when a few friends and I wanted to start a project that was partially web based. In terms of scalability, node.js was the hip new thing and one of us already had experience with it, so we first went with that. However, using JavaScript meant that we had to rewrite and maintain parts of the D code in JavaScript, and more importantly, programming with node.js was really awful due to the callback-based API.

Then I remembered D's "Fiber" class, which was inherited from Tango's runtime during the D1->D2 transition. This was the missing piece of the puzzle to an I/O system in D that could be made better in almost every aspect. So we decided that we'd put some concentrated effort into this to make the development of the project more bearable. Two months later, the first public alpha version was born.

- You call vibe.d "a high-performance asynchronous I/O, concurrency and web application toolkit:" how does that differ from a web framework like ruby on rails? Why the focus on I/O and concurrency before web applications?

- The initial focus was on getting a generic framework running that could be used as the basis for event based I/O, but also had means to do general thread-based concurrency without interfering with I/O and integrated well with the GUI event loop. And although the HTTP server component has always been part of the library, all of the higher-level web functionality was only added later, step by step, to fill the gaps needed for creating larger-scale web services.

Today, with the inclusion of high-level declarative web and REST frameworks, the emphasis has finally shifted more towards web development. The main reason for going this way and not creating a separate high-level library is that there are loose ongoing plans to eventually integrate the I/O and concurrency abstraction directly into the D standard library, and this way there will still be enough meat left to continue the library in its current form. But the time may actually be ripe for changing that slogan.

- How did DUB come to be? Any other info on its current use and success would be worthwhile.

- DUB was born as a spin-off of vibe.d. It included a small package manager, the vibe.d package manager "VPM", specifically meant to write vibe.d based extensions and applications, relying on rdmd exclusively to build everything. At that time we didn't want to set the goals too high and decided against positioning it as a full D package manager. VPM, in its half year of existence, hadn't been very successful in terms of adoption (about a dozen public packages, most of them our own), but after growing calls for a package manager within the D community, we were finally convinced that we could serve the language ecosystem better by making it a generic tool for D applications and libraries. So we basically started to rip out all vibe.d specifics as far as possible, added proper generic substitutes, and released a first version of DUB.

People soon started to add their own packages, but it took another six months until we switched the address of the package registry from registry.vibed.org to code.dlang.org and things really took off. Since then we've had a steady growth of packages going from 40 at the time of the switch to almost 500 as of now. Taking only this measure, D2, after the big Phobos-Tango-D2 exodus and a long struggle, is now more successful than D1 during its best times (dsource.org had about 260 projects listed). My hope is that the growth and use of packages now gets another acceleration due to the new prominent links to code.dlang.org on the D homepage, so that one day we'll have a vibrant one-stop shop for all kinds of libraries and tools, similar to what more popular languages have accumulated today (Ruby has over 100,000 packages, node.js has crazy 126,000 packages - not sure if we'd want to go that far).

In terms of future direction, while the current feature set of DUB, with only little exceptions, has proven to be very successful for supporting an ecosystem of pure D packages, the next goal for DUB is now to get better support for more advanced use cases, such as mixed-language projects (e.g. C++ and D) or more complicated build processes.

- Could you talk about your experiences with D in building vibe.d, ie things you liked about D when writing vibe.d and things you had to work around?

- In general, D has always been very pleasant to work with, including when developing vibe.d. I have never experienced this level of productiveness for medium to large scale projects with any other language so far. However, hitting compiler bugs and regressions of all sorts was common, and finding workarounds/reproducible cases and reporting bugs at times diminished a lot of the productivity gains. However, although there is still something that breaks with almost any new beta release of DMD, this has gotten much better over time and isn't so much of an issue today anymore. It can still be a little difficult to maintain compatibility with a range of D compilers (vibe.d tries to support 2.065-2.067 ATM), especially since using new language features can be very tempting!

Speaking of which, one of the most useful additions to D would be things like proper "scope" and isolated (a.k.a. "unique") references. Those would enable the memory-safe use of faster programming paradigms, such as using stack-allocated classes, or accessing objects from different threads without the need for costly locks or atomic operations. These are things that vibe.d could greatly benefit from, but memory safety is too important for network services to go this route without language-based verification.

- Please tell us a bit about yourself: who you are, what programming languages you used before D, and your experience first discovering and using D.

I grew up and still live in northern Germany near the city of Lübeck, which is also where I graduated in bioinformatics a few years ago. My first programming experiences began in my early teenager times when I got my first 386DX, running DOS/Windows 3.1. I discovered qbasic.exe soon after playing around with some simple batch files and began creating all kinds of little GUI and 3D-Graphics based DOS applications.

The experience of the direct feedback back then got me hooked. After some tinkering with TurboBasic - boy, was that fast - and another BASIC based Windows IDE (CA Realizer) around '95, a friend of mine gave me a CD with Borland C++ 5.0, which I then discovered step by step in a completely naive and trial/error based way (What is a "pointer"? Why does it crash now?). The next thing I remember are some bad memories involving Visual C++ 6 and tons of cross compiler compatibility macros, and discovering all of the details of the C++ standard that I wouldn't ever have wanted to know about.

In these times, I created a big number of small programs, discovering the darkest corners of the Windows API, network protocols, hardware interfaces, OpenGL and other random things. I also tried out a lot of other programming languages, including most of the popular scripting languages and a short trip to OCaml, but I've mainly stuck to C and C++ for actual projects (still preferring C over C++ due to many factors). W32DASM was also one of my favorite tools at the time, without going into details of what kind of 1337 things that was used for, but it also helped me learn the details of x86 assembly.

Then, during my time at the university, I worked for Nik Software, a former company which specialized in software for professional photographers, later bought out by Google. There I managed the creation of the company-wide cross-platform core library, providing higher level concepts needed for almost all products. This included an image-filtering framework capable of transparently using multi-core CPUs and GPUs to filter images of arbitrary sizes and a custom GUI framework, which allowed us to maintain a single code base for all platforms, while having the ability to implement advanced GUI concepts that were more or less impossible with other GUI frameworks at the time.

For my masters thesis back in 2011, I created a compiler in D for an image filtering-specific DSL, which was able to translate to GPU shader code (GLSL or HSL), to C++ code and, thanks to LLVM, directly to machine code at runtime. It was then integrated into the company's core library to form the basis for new filters. This was the fist piece of software in the company which was written in D (controlled through an extern(C) interface). The language made this a very pleasant and an extremely efficient endeavor (apart from some friction caused by the C interfaces to the host application and to the LLVM libraries). I intentionally tried to keep the D code simple, without excessive use of advanced meta-programming and other "bleeding edge" features, so that the chance of running into compiler bugs was as low as possible (there have still been quite some of these, and getting 64-bit builds to work on Windows also still was a challenge at the time).

- Do you make a living with Rejected Software and writing D? What kind of business are you into?

- I used to make a living by doing various kinds of software development contract work for some companies, which I still do, but only to a small extent. Topics include low-level development support, accounting software, image processing, web development, graphics creation and more. Some of the projects are written in D, but most are still implemented with other languages as per the request of the customer.

Today, I try to focus more and more on getting our products ready instead, so that one day those will hopefully be able to generate enough income alone. Unfavorable for these plans, offspring and moving to a house has consumed so much time recently that I had to suspend most of my work, including the open source projects. But, setting aside any unforeseen events, I'm hoping the best for getting back to full speed this year.

- What do you love and/or hate about D? This is more general than the specific vibe.d-related question above.

- I basically love the initial (D1) design choices - the clean and easy to parse syntax (field access, templates, slices, foreach etc.), the module system and the low compile times. Also D's meta-programming abilities, including CTFE and the more recent support for user-defined attributes, are superb and, I'm sure other D programmers know this feeling as well, it's sometimes really hard for me now to force myself into doing things with certain other languages that could have been done much more efficiently in D, even when talking just about a few lines of code more or less.

But the thing I probably dislike most about the language is the at-times strong friction against fundamental improvements in favor of keeping backwards compatibility (which doesn't happen anyway). I believe that the language has to keep moving fast towards "perfection" to stay (or become) successful. Other languages show ways to mitigate the effects of breaking changes, starting with proper deprecation paths and continuing with automatic code-fixing tools or even GitHub bots that file automatic pull requests with fixes for all public repositories.

An example of a simple but fundamental issue are the defaults of the built-in attributes. I think some of them, for historical or compatibility reasons, are currently simply the wrong way around (pure, @safe, final and scope should really all be enabled by default, with scope providing recursive guarantees) and using them properly completely destroys the initial idea of having a clean language syntax. It's sometimes really sad to see modern idiomatic D code degrading into a mess of attributes and contract syntax noise. After all, a clean syntax used to be one of the key selling points. Attribute inference helps somewhat with this, but that only applies to template functions and also makes tracking down attribute mismatch issues more difficult.

Upcoming events

Learn more about D

To learn more about D and what's happening in D: