English

The nature of open-source software development is to be public. The source code is public, the bug reports are public, the feature requests are public, the discussion is public. Which means everyone can chime in and participate in them. It looks great on paper, but that's not necessarily a good thing in practice. In any case, that calls for some ground rules if we want to turn it into a productive thing.

For context, you may want to read :

1. The designer and the drilling machine
2. Design by committee will not save FLOSS
3. Contributing to Ansel

Introduction

We live in a world where the volume of information and communication has become overwhelming and humans don't have the bandwidth to process all of it. Endless threads and unregulated conversations are actively harming communication by diluting important information and exhausting the reader. Discussion is solely meant to reach an understanding and proceed to actionable decisions. Any other form of communication is called informal chat, it's fine but it should happen on dedicated channels, aside from the problem solving process.

What is a problem ?

Photographs have a life cycle starting when you press the camera shutter button and ending when the final image is printed, saved for archiving or otherwise published. This requires work using technological appliances that all have their flaws. Problems are anything making this work more time-consuming, complicated or unreliable than it should be.

Now, "what it should be" is in the eyes of the beholder and is limited by what is possible, which is where a technical reality check is needed. "Possible" implies 3 things :

1. it can be done at all,
2. it can be done with currently available resources,
3. it can be maintained in the future with currently available resources.

The rule of thumbs here is what is possible has most likely already be done, with some exceptions where nobody thought of something simple. The maintainability is the most overlooked aspect of the 3 because it is about a distant and fuzzy future. The industry is full of better technologies that were abandonned because their maintenance costs were simply too high.

How to define a problem ?

Clients, customers and users all suck at defining their own problems. It's just how it is : they are too close from them to fully see them, and they often lack the technical knowledge to properly analyse them. In software matters, clients, customers and users also only see the GUI part of things, with no idea how things are wired behind. So all they can think of is "button to do this", "graphs to show that", "hacky way of slightly modifying something existing". Those GUI-based ways of defining problems always promote solutions that lack generality and favour more complexity, because thinking in terms of graphical elements bypasses abstract thinking, and abstraction is the only way of simplifying complex matters (even though most people have an hard time with it).

Problems should be defined in terms of goal to achieve or task to complete. "I want my picture to look more X or less Y", "I need to export pictures for Z", "I find myself having to repetitively do W all the time" are good ways of defining problems. "X is broken", "Y should be like some other software", "I don't like Z", "I need W (designed for C) to also do A and B" are bad ways.

What makes a good problem definition ? Keep in mind that the problem you see is very often at the end of the branch. If we focus too much on the end, we will miss the stem and the root of the problem. Finding and solving the root problem calls for more simple solutions and often fixes other end problems cleanly and in a minimalist way. Focusing on end problems triggers lots of contextual patches that increase complexity (it's what the Darktable's idiots do). A good problem definition is one that helps finding the root problem. Focusing on the goal instead of the implementation or the technical details is the best way to achieve it. Getting dazzled by technicalities is the surest way to get lost.

Stick to the problem

The number one mistake is to suggest a solution alongside the problem definition, hoping it will help and speed things up. On the contrary, this is actively harmful.

Rigour and due process take some non-compressible time and rushing things will usually cost more time later. Proposing solutions and technologies, drawing sketches or even showing prototypes before the problem is fully understood is actually biasing the solution too early in the process and will only lead people into dead-ends. The reason is humans are lazy (as they should), so whatever half-baked solution available right now will always be more appealing than having to sit for longer on our hands and keep thinking. Again, that usually leads to future time losses when the sub-optimal solution (and its overlayers) will need to be changed or redone.

Contextualize the problem

Design and engineering don't solve technical problems (common misconception), they solve people's problems with technology. Which means there is no solution before we understand who is having the problem. It's often hard for non-tech people to gage what the proper amount of background info is. Too much is tiresome to read and dilutes the important bits, too little leaves too many open questions.

Proper context includes : 

• what kind of photography you do and how (e.g. wedding -> large amounts of pictures processed in batch, art -> small amounts of individually fine-tuned pictures),
• how comfortable you are with computers,
• the version of the software you are using and where you got it,
• the operating system you are using (name, version, drivers)
• the hardware you have (especially the GPU brand and generation if any).

One thread = One topic

For the purpose of efficient communication but also for project management and for later indexing by search engines, each discussion thread, bug report or feature request should have exactly one topic. Open new threads for other topics.

Compromises are not solutions

Not sure why the ability to compromise is seen as quality in today's world, but compromise is a loose-loose relationship where we resolve conflicts by meeting half-way in losses. Say you want to go the movie theater see a motion picture and your significant other wants to go to the restaurant. Compromise is eating half the meal and seeing half of the movie : everybody is guaranteed to have a bad time.

Now, if you investigate motives and internal goals, you may have a chance of finding out that your significant other wants the restaurant to avoid the task of cooking and cleaning up the dishes, and you want the movie to relax and unwind. You can achieve both your goals by ordering food to take out and watching Netflix at home. No compromise needed here.

Whenever you have to resort to compromise, it usually means you failed at investigating the root goals and… at solving the root problem. Compromises are typically born from a design process stopped too early, looking only at the tip of the iceberg.

Working towards a solution

Let the designers do their work, thanks. It's a job that requires skills, knowledge, method and rigour. The open-source world is already full of people who thought they could do it too, and that's why Linux runs on 2 % of desktop computers.

------

La nature du logiciel open-source est publique. Le code source est public, les rapports de bugs sont publics, les demandes de fonctionnalités sont publiques, la discussion est publique. Ceci signifie que tout le monde peut participer. Ça paraît bien sur le papier, mais c'est loin d'être nécessairement une bonne chose en pratique, ce qui demande certaines règles de base pour que ça devienne effectivement une bonne chose.

Pour le contexte, vous devriez lire (en anglais) :

1. The designer and the drilling machine
2. Design by committee will not save FLOSS
3. Contributing to Ansel

Introduction

Nous vivons dans un monde où le volume d'informations et de communication est devenu assommant et les humains n'ont pas les capacités cognitives de tout traiter. Les fils de discussions interminables et les conversations non-régulées endommagent activement la communication en diluant les informations importantes et en épuisant le lecteur. La discussion est exclusivement faite pour parvenir à une entente et poursuivre vers des décisions actionnables. Toute autre forme de communication relève du chat informel, ce qui est acceptable mais doit intervenir sur des canaux dédiés, en marge de la résolution de problèmes.

Qu'est-ce qu'un problème ?

Les photographies ont un cycle de vie qui commence quand vous appuyez sur le bouton du déclencheur et qui termine quand l'image finale est imprimée, archivée ou autrement publiée. Ceci requiert du travail utilisant des appareils technologiques qui ont chacun leurs défauts. Les problèmes sont tout ce qui rend ce travail plus consommateur de temps, compliqué ou non-fiable que ce qu'il devrait être.

Maintenant, "ce qu'il devrait être" est subjectif et limité par ce qui est possible, et c'est là qu'un retour à la réalité technique est nécessaire. "Possible" implique 3 choses : 

1. c'est réalisable de manière générale,
2. c'est réalisable en particulier avec les ressources actuellement disponibles,
3. ça peut être maintenu dans le futur avec les ressources actuellement disponibles.

La règle empirique ici est que ce qui est possible a probablement déjà été fait, à quelques exceptions près où personne n'a pensé à des choses simples. La maintenabilité est l'aspect le plus négligé des 3 car il concerne un futur distant et flou. L'industrie est pleine de technologies meilleures qui ont été abandonnées parce que leurs coûts de maintenance était simplement trop élevés.

Comment définir un problème ?

Les clients et les utilisateurs sont simplement nuls pour définir leurs propres problèmes. C'est comme ça : ils sont trop proches de leurs problèmes pour les voir en entier et manquent souvent des connaissances techniques pour les analyser correctement. Sur les sujets logiciels, les clients et utilisateurs ne voient que la partie interface graphique, sans aucune idée de la façon dont les choses sont câblées à l'arrière. Tout ce à quoi ils pensent est donc "un bouton pour faire ci", "un graphique pour montrer ça", "des manières sales de bidouiller légèrement l'existant". Ces façons de définir les problèmes basées sur l'interface encouragent toujours des solutions qui manquent de généralité et introduisent plus de complexité, parce que penser en termes d'éléments graphiques court-circuite la pensée abstraite, et l'abstraction est la seule façon de simplifier des problèmes complexes (même si la plupart des gens détestent ça).

Les problèmes doivent être définis en terme de buts à atteindre ou de tâche à accomplir. "Je veux que mon image paraisse plus X ou moins Y", "j'ai besoin d'exporter des images pour Z", "je me trouve obligé de répéter l'opération Z tout le temps" sont de bonnes façons de définir des problèmes. "X est cassé", "je n'aime pas Z", "Y devrait être comme dans un autre logiciel", ou "j'ai besoin que W (conçu pour C) fasse aussi A et B" sont des mauvaises manières.

Qu'est ce qui fait qu'une définition de problème est bonne ? Gardez à l'esprit que le problème que vous voyez est très souvent l'extrémité de la branche. Si l'on se concentre trop sur l'extrémité, on va manquer la tige et la racine du problème. Trouver et résoudre la racine du problème appelle des solutions plus simples et règle souvent d'autres problèmes terminaux de manière propre et minimaliste. Se concentrer sur les problèmes terminaux entraîne de nombreux patchs contextuels qui augmentent la complexité (c'est ce que font les idiots de Darktable). Une bonne définition du problème aide à trouver la racine du problème. Se concentrer sur le but plutôt que sur l'implémentation ou les détails techniques est le meilleur moyen d'y parvenir. Se laisser aveugler par les technicalités est le plus sûr moyen de se perdre.

S'en tenir au problème

La principale erreur est de suggérer une solution en même temps que la définition du problème, en espérant aider et faire gagner du temps. Au contraire, c'est activement préjudiciable.

La rigueur et le respect des procédures prend un temps non compressible et les bâcler pour gagner du temps va habituellement coûter plus de temps plus tard. Proposer des solutions et des technologies, dessiner des croquis ou même montrer des prototypes avant que le problème ne soit complètement compris revient à biaiser la solution trop tôt dans le processus et ne fera que mener les gens dans des impasses. La raison est que les humains sont paresseux (ce qui est normal), si bien que toute demi-solution disponible immédiatement sera toujours plus séduisante que s'asseoir sur ses mains plus longtemps pour continuer à réfléchir. Encore une fois, ceci ne fait qu'amener des pertes de temps futures, quand la solution sous-optimale (et ses surcouches) devra être changée ou refaite.

Contextualiser le problème

La conception et l'ingéniérie ne résolvent pas des problèmes techniques (idée fausse répandue), elles résolvent des problèmes que les gens ont avec la technologie. Ceci signifie qu'il n'y a pas de solution tant qu'on ne comprend pas qui a le problème. Il est souvent difficile pour les personnes non techniques d'évaluer la quantité adéquate d'informations générales à donner. Trop est fatiguant à lire, pas assez laisse trop de questions ouvertes.

Un contexte adéquat inclue :

• quel type de photographie vous pratiquez et comment (ex : mariage -> grandes quantités d'images à traiter en lot, art -> petites quantités d'images ajustées individuellement),
• à quel point vous êtes à l'aise avec les ordinateurs,
• la version du logiciel utilisé et où vous l'avez obtenu,
• le système d'exploitation que vous utilisez (nom, version, pilotes)
• le matériel utilisé (en particulier la marque et la génération de la carte graphique s'il y a lieu).

Une discussion = Un sujet

Dans l'objectif de communiquer efficacement mais aussi pour la gestion de projet et l'indexation ultérieure par des moteurs de recherche, chaque fil de discussion, rapport de bug ou demande de fonctionnalité doit avoir exactement un sujet. Ouvrez de nouveaux fils pour d'autres sujets.

Les compromis ne sont pas des solutions

Je ne sais pas pourquoi la capacité à faire des compromis est vue comme une qualité dans le monde actuel, mais un compromis est une relation perdant-perdant où l'on résout les conflits en se rencontrant à mi-chemin dans les pertes. Imaginez que vous souhaitiez allez au cinéma voir un film et que votre tendre moitié veule aller au restaurant. Le compromis est de manger la moitié du repas et de voir la moitié du film : personne ne va apprécier l'expérience.

Maintenant, si vous investiguez les motivations et les buts internes, vous pourriez avoir une chance de découvrir que votre moitié veut le restaurant pour éviter de cuisiner et de faire la vaisselle, et que vous voulez le film pour vous détendre et vous évader. Vous pouvez atteindre chacun votre but en commandant un repas à emporter et en regardant Netflix à la maison. Compromis évité.

À chaque fois que vous devez vous résoudre à faire un compromis, cela signifie habituellement que vous avez échoué dans l'investigation des buts sources et… dans la résolution de la racine du problème. Les compromis sont typiquement une démarche de conception arrêtée trop tôt, en se limitant à la partie émergée de l'iceberg.

Travailler sur une solution

Laissez les concepteurs faire leur travail. C'est un métier qui requiert des compétences, des connaissances, de la méthode et de la rigueur. Le monde open-source est déjà plein de gens qui ont cru qu'ils pouvaient le faire aussi, et c'est la raison pour laquelle Linux tourne sur 2 % des ordinateurs de bureau.

If you come from Darktable, you may be used to this in the darkroom:

while Ansel offers you this:

This is no accident, and tt's time to explain why, and why this will not be extended with customization options.

Images are born from pipelines

A pixel pipeline is a sequence of filters in which pixels are processed to end on a medium. Photoshop calls those filters layers, abiding by a methaphor inherited from paper and matte painting. Da Vinci Resolve, Blender, Natron, etc. calls them nodes, abiding by a methaphor grounded in directed graphs and flowcharts, best known to engineers. Both have a way of showing how those filters are organized, either with a layer stack or with the node graph (aka flowchart).

The important part is, order matters.

A brief history of bad design

Darktable calls those filters modules. But "modules" refers to the implementation/programming logic : each module is coded separately, using an uniform API, and doesn't know about the other modules. The pipeline itself doesn't know anything about the internals of modules, it only wires the inputs and the outputs. It's a clean way of developing, but it's completely irrelevant to the end-user.

The problem is Darktable has 2 kinds of modules : 

• the lighttable modules (and the modules in the left panel of the darkroom), which are arbitrary toolboxes and therefore purely GUI content,
• the darkroom modules, which are both a pixel filter located somewhere in the pipeline, and also a GUI toolbox.

And those different modules look exactly the same…

That's 3 mistakes here : 

1. naming a GUI object by its technical implementation name instead of its purpose,
2. naming and representing 2 different objects the same way,
3. failing to represent the order of the modules in a clear, hierarchical way.

As a result, many users still consider all modules to be arbitrary toolboxes, and have asked for years ways to reorder them arbitrarily in the window, which a spineless (lack of) technical leadership gave to them, in the shape of a terribly-coded (3500 lines of code, subtly broken) and overcomplicated module group, eating 3 % of your CPU even when you don't interact with the application, for as long as your darkroom sits idle.

The cherry on the sunday is the groups are labelled by icons, for the sake of compactness, but those icons are absolutely cryptic and only the long-time users pretend to know what they represent (I draw the one representing light rays going through a thin lens, that people take for an UFO – never again).

Good workflows are pipeline-aware

It's been 3 years that I'm paid by users to explain them the ins and outs of the software, and answering the same question over and over : where to start a workflow and how to unroll it. What still strikes me is that people with a master's degree, who read the doc and watched most of my videos, are still unable to start an image editing workflow by themselves. That's screaming bad design.

If you open Photoshop, the layers stack upon each other pretty intuitively. We all worked with layers for art projects in primary school. It wouldn't strike you to start working on the bottom-most layer after you put some new stuff on top. Well, the 70-something modules of Darktable, organized in tabs by theme, in a way that doesn't account for the pipeline nor for the workflow, are guaranteed to deter the newcomers and to promote bad habits among the old timers.

Sane workflows are pipeline-aware, which means the order in which you tune the filters should be defined by where those filters sit in the pipeline. But I say pipeline-aware, and not pipeline-defined, because the beginning and the end of the pipeline (properties of the scene and of the display) should be set first, as to have a good look over what we do in-between. Especially if you are going to manipulate HDR signals on an SDR display, you need to put on your HDR sunglasses first to view your signal in SDR. But what you see is not what there is in your pipeline. Hence why the workflow doesn't follow 1:1 the pipeline, but still stick pretty close to it.

So how do you know when to drift from the pipeline order ? Well, you book a session with me for the demo. But there is another solution…

In any case, offering users more options to customize the UI (and perhaps re-enforcing the initial misconception of modules being only GUI boxes) is not going to solve it. It's actually giving people more options to harm themselves. What you want and what is good for you…

Re-examining the problem

While Darktable has degraded into a playground for geeks where new means better and every problem calls for more funny code, Ansel is about solving simple problems the simple way, as to produce a reliable workhorse. So let's start again from the top.

We have 70 modules. Though Ansel has deprecated a fair deal of them, there are still "too many", in the sense that they are all useful for some purpose but you don't need them all the time. Also the screen real-estate is limited and we can't have all of them displayed at the same time. And even if we could, presenting and Airbus dashboard to your average photographer wouldn't be nice.

So we need to chose what modules to display at what time. Emphasis on time.

Unrolling the time axis

It seems only natural that the time axis would be splitted into workflow steps. So the selection of all visible modules at a given time matches the one that you will need right now and in the next minutes. Moving on to the next workflow step, you move on into the GUI and change the view. This draws a linear path to follow, to get some structure and guidance out of the apparent clutter. GUI are not only meant to expose controls, they are also meant to teach, communicate and advertise the available possibilities.

Implementation

Principle

1. Workflow steps == module tabs. Those tabs have textual names, which might take more space but you don't have to read a doc and/or guess what they mean.
2. The first and last tabs are special : they show respectively the list of enabled modules (pipeline) and the whole list of available modules (all).
3. Not all tabs are immediately visible, depending on sidepanel width, which is fine because you are going to follow them from left to right in sequence,
4. Inside tabs, modules are organized as layers in the pipeline order, that is from bottom to top. That's how you should set them.

TL;DR: follow GUI order from left to right, and bottom to top (because it's layers), and you have your workflow without tedious doc reading.

The darkroom modules can be reordered in any tab by holding Ctrl+Shift while doing a drag-and-drop with the mouse, over the module headers. Be aware that this reorders modules in the pipeline too, it is not to be used as a GUI convenience. It is best to this in the "pipeline" or "all" tabs, where you have a full look over the pipeline content.

Linear navigation

With the mouse, tabs can be navigated by:

• clicking the ones that are visible,
• clicking on the arrows to reveal the previous/next tabs (visible or hidden),
• right-clicking on any tab to reveal a contextual menu containing the list of all tabs,
• scrolling previous/back on the tabs bar to switch tabs. Once you reached the end, it will cycle back with an "hard push" ensuring it's really what you want to do.

Then modules can be navigated by scrolling them in the sidebar, the scroll event is captured by sliders only after a previous interaction was done with them. Unfolding modules automatically scroll the bar if needed, as to display them in full if possible, or at least to align their upper border with the viewport.

With the keyboard, tabs can be navigated by Ctrl+Tab and Ctrl+Shift+Tab to switch between next/previous tabs. This will also cycle back when you reached the end of tabse

Then modules can be navigated with Page Up/Down to move to previous/next module. This automatically unfolds each selected module while folding all the others, and aligns the current module vertically as to try and fit it entirely in the screen.

This sequential way of accessing next/previous module is meant to solve the terrible UX of Darktable where you have to assign exactly one shortcut per module to unfold it, and then have to remember all those special shortcuts while being unable to re-use them in other parts of the software.

Transversal navigation

If you have already been trapped in an Ikea, you know the benefits of finding transverse pathes instead of following the intended path.

You can search modules by their names or internal aliases with the search box, which responds to the Ctrl+F shortcut. The search results can also be accessed in sequence by Page Up/Down keys.

Remember that the first tab shows the whole pipeline in its order of application.

Replacing favourite modules

The current design has no way to define favourite modules in a special tab. I don't see the point of adding bloat to solve the issue of having bloat.

For those special modules, you can assign shortcuts to the "show" (aka open, show, display, unfold) or "enable" events (aka activate). Go to the Edit menu, then at the bottom click Keyboard shortcuts, then with the special cursor you got, click on your soon-to-be favorite module header (on its name). Example here with the exposure module :

By default, you will be prompted with the "show" event, in the element column (another label I should change). You can change it for the "enable" or "instance" (aka instanciate) event. The effect doesn't matter for this one, I haven't tested in which case it is used and the whole thing is a tangled mess anyway.

In any case, these shortcuts will instantly bring you to your favourite modules without polluting more GUI real-estate.

Conclusion

This doesn't solve the issue of modules beeing called something that don't interest users, and of image processing modules looking the same as non-image processing ones. I have some ideas regarding this, but that will be for another week.

Sidenotes

Many other tools previously hidden in cryptic icon-buttons have been merged into the global menu. This menu can be unfolded by hitting Alt followed by the mnemonic letter of the menu (which will get underscored once hitting Alt). Once unfolded, the menu are navigable with arrow keys.

Return to the lighttable is now mapped to the Escape key. In lighttable, the text search is also mapped to Ctrl+F. Browsing pictures can be done with Arrow keys, selection with Space key, and opening a picture in darkroom can be done by hitting Enter key.

This means the application is now almost entirely navigable with the keyboard without having to remember shortcuts. Those shurtcuts are displayed anyway in the menu, right of the entries.

The table of all shortcuts can now be found in the Help menu, previously it was only accessible… through a shortcut.

I accidentally discovered that the Linux build script used a "package" build, meaning the CPU optimizations are limited to generic ones in order to produce portable binaries that can be installed on any x86-64 platform. By "using", I mean the package build was not explicitely disabled, so it was enabled by default.

Anyway, this is now disabled by default, since the actual packages (.exe and .appimage) are not built through that script, which is primarily meant to help end-users. To get the previous behaviour back, you would need to run:

$ sh build.sh --build-package --install --sudo

Not using the package build option may increase performance on CPU by 20 to 30 % depending on your hardware, thanks to platform-specific optimizations.

I have also introduced a new argument that will launch the Git updating commands that users seem to forget all the time. There is a caveat, though : updating the source code by calling Git from within the script doesn't update the script for the current run, so this method doesn't work when the script itself is modified. The argument to update the source code and the submodules (Rawspeed, Libraw) :

$ sh build.sh --update --install --sudo

I have also modified the internals of that script in order to automatically : 

• update the Lensfun database of lenses,
• add a global system shortcut (.desktop file) so the software will be globally available from the app menus,
• add a global system command so the ansel is globally available from the terminal.

The goal of all these changes is obviously to make it more user-friendly to use a self-built version of the software, allowing to improve performance, especially for computers without GPU. The one-pit-stop command would be :

$ sh build.sh --update --install --sudo --clean-all

But of course, you will need to run the Git update manually one last time before, to update the script itself :

$ git pull --recurse-submodule

Alternatively, you can directly download the build script, and replace the old build.sh one at the root of the source code directory.

#dev