Debian is a trademark of Software in the Public Interest, Inc. This site is operated independently in the spirit of point three of the Debian Social Contract which tells us We will not hide problems.

Feeds

October 23, 2025

hackergotchi for Steinar H. Gunderson

Steinar H. Gunderson

Modern perfect hashing

Wojciech Muła posted about modern perfect hashing for strings and I wanted to make some comments about my own implementation (that sadly never got productionized because doubling the speed compared to gperf wasn't really that impactful in the end).

First, let's define the problem, just so we're all on the same page; the goal is to create code that maps a known, fixed set of strings to a predefined integer (per string), and rejects everything else. This is essentially the same as a hash table, except that since the set of strings is known ahead of time, we can do better than a normal hash table. (So no “but I heard SwissTables uses SIMD and thus cannot be beat”, please. :-) ) My use case is around a thousand strings or so, and we'll assume that a couple of minutes of build time is OK (shorter would be better, but we can probably cache somehow). If you've got millions of strings, and you don't know them compile-time (for instance because you want to use your hash table in the join phase of a database), see this survey; it's a different problem with largely different solutions.

Like Wojciech, I started splitting by length. This means that we can drop all bounds checking after this, memcmp will be optimized by the compiler to use SIMD if relevant, and so on.

But after that, he recommends using PEXT (bit extraction, from BMI2), which has two problems: First, the resulting table can get quite big if your input set isn't well-behaved. (You can do better than the greedy algorithm he suggests, but not infinitely so, and finding the optimal mask quickly is sort of annoying if you don't want to embed a SAT solver or something.) Second, I needed the code to work on Arm, where you simply don't have this instruction or anything like it available. (Also, not all x86 has it, and on older Zen, it's slow.)

So, we need some other way, short of software emulation of PEXT (which exists, but we'd like to do better), to convert a sparse set of bits into a table without any collisions. It turns out the computer chess community has needed to grapple with this for a long time (they want to convert from “I have a \ on \ and there are pieces on relevant squares \, give me an index that points to an array of squares I can move to”), and their solution is to use… well, magic. It turns out that if you do something like ((value & mask) * magic), it is very likely that the upper bits will be collision-free between your different values if you try enough different numbers for magic. We can use this too; for instance, here is code for all length-4 CSS keywords:

   static const uint8_t table[] = {
        6,   0,   0,   3,   2,   5,   9,   0,   0,   1,   0,   8,   7,   0,   0,
   };
   static const uint8_t strings[] = {
       1,   0, 'z', 'o', 'o', 'm',
       2,   0, 'c', 'l', 'i', 'p',
       3,   0, 'f', 'i', 'l', 'l',
       4,   0, 'l', 'e', 'f', 't',
       5,   0, 'p', 'a', 'g', 'e',
       6,   0, 's', 'i', 'z', 'e',
       7,   0, 'f', 'l', 'e', 'x',
       8,   0, 'f', 'o', 'n', 't',
       9,   0, 'g', 'r', 'i', 'd',
      10,   0, 'm', 'a', 's', 'k',
   };

   uint16_t block;
   memcpy(&block, str + 0, sizeof(block));
   uint32_t pos = uint32_t(block * 0x28400000U) >> 28;
   const uint8_t *candidate = strings + 6 * table[pos];
   if (memcmp(candidate + 2, str, 4) == 0) {
     return candidate[0] + (candidate[1] << 8);
   }
   return 0;

There's a bit to unpack here; we read the first 16 bits from our value with memcpy (big-endian users beware!), multiply it with the magic value 0x28400000U found by trial and error, shift the top bits down, and now all of our ten candidate values (“zoom”, “clip”, etc.) have different top four bits. We use that to index into a small table, check that we got the right one instead of a random collision (e.g. “abcd”, 0x6261, would get a value of 12, and table[12] is 7, so we need to disambiguate that from “flex”, which is what we are actually looking for in that spot), and then return the 16-bit identifier related to the match (or zero, if we didn't find it).

We don't need to use the first 16 bits; we could have used any other consecutive 16 bits, or any 32 bits, or any 64 bits, or possibly any of those masked off, or even XOR of two different 32-bit sets if need be. My code prefers smaller types because a) they tend to give smaller code size (easier to load into registers, or can even be used as immediates), and b) you can bruteforce them instead of doing random searches (which, not the least, has the advantage that you can give up much quicker).

You also don't really need the intermediate table; if the fit is particularly good, you can just index directly into the final result without wasting any space. Here's the case for length-24 CSS keywords, where we happened to have exactly 16 candidates and we found a magic giving a perfect (4-bit) value, making it a no-brainer:

  static const uint8_t strings[] = {
     95,   0, 'b', 'o', 'r', 'd', 'e', 'r', '-', 'b', 'l', 'o', 'c', 'k', '-', 's', 't', 'a', 'r', 't', '-', 'w', 'i', 'd', 't', 'h',
     40,   0, '-', 'w', 'e', 'b', 'k', 'i', 't', '-', 't', 'e', 'x', 't', '-', 'o', 'r', 'i', 'e', 'n', 't', 'a', 't', 'i', 'o', 'n',
    115,   1, 's', 'c', 'r', 'o', 'l', 'l', '-', 'p', 'a', 'd', 'd', 'i', 'n', 'g', '-', 'b', 'l', 'o', 'c', 'k', '-', 'e', 'n', 'd',
    198,   2, '-', 'w', 'e', 'b', 'k', 'i', 't', '-', 't', 'r', 'a', 'n', 's', 'f', 'o', 'r', 'm', '-', 'o', 'r', 'i', 'g', 'i', 'n',
    225,   0, '-', 'i', 'n', 't', 'e', 'r', 'n', 'a', 'l', '-', 'o', 'v', 'e', 'r', 'f', 'l', 'o', 'w', '-', 'b', 'l', 'o', 'c', 'k',
    101,   2, '-', 'w', 'e', 'b', 'k', 'i', 't', '-', 'b', 'o', 'r', 'd', 'e', 'r', '-', 'e', 'n', 'd', '-', 's', 't', 'y', 'l', 'e',
     93,   0, 'b', 'o', 'r', 'd', 'e', 'r', '-', 'b', 'l', 'o', 'c', 'k', '-', 's', 't', 'a', 'r', 't', '-', 'c', 'o', 'l', 'o', 'r',
    102,   2, '-', 'w', 'e', 'b', 'k', 'i', 't', '-', 'b', 'o', 'r', 'd', 'e', 'r', '-', 'e', 'n', 'd', '-', 'w', 'i', 'd', 't', 'h',
    169,   1, 't', 'e', 'x', 't', '-', 'd', 'e', 'c', 'o', 'r', 'a', 't', 'i', 'o', 'n', '-', 's', 'k', 'i', 'p', '-', 'i', 'n', 'k',
    156,   0, 'c', 'o', 'n', 't', 'a', 'i', 'n', '-', 'i', 'n', 't', 'r', 'i', 'n', 's', 'i', 'c', '-', 'h', 'e', 'i', 'g', 'h', 't',
    201,   2, '-', 'w', 'e', 'b', 'k', 'i', 't', '-', 't', 'r', 'a', 'n', 's', 'i', 't', 'i', 'o', 'n', '-', 'd', 'e', 'l', 'a', 'y',
    109,   1, 's', 'c', 'r', 'o', 'l', 'l', '-', 'm', 'a', 'r', 'g', 'i', 'n', '-', 'i', 'n', 'l', 'i', 'n', 'e', '-', 'e', 'n', 'd',
    240,   0, '-', 'i', 'n', 't', 'e', 'r', 'n', 'a', 'l', '-', 'v', 'i', 's', 'i', 't', 'e', 'd', '-', 's', 't', 'r', 'o', 'k', 'e',
    100,   2, '-', 'w', 'e', 'b', 'k', 'i', 't', '-', 'b', 'o', 'r', 'd', 'e', 'r', '-', 'e', 'n', 'd', '-', 'c', 'o', 'l', 'o', 'r',
     94,   0, 'b', 'o', 'r', 'd', 'e', 'r', '-', 'b', 'l', 'o', 'c', 'k', '-', 's', 't', 'a', 'r', 't', '-', 's', 't', 'y', 'l', 'e',
    196,   2, '-', 'w', 'e', 'b', 'k', 'i', 't', '-', 't', 'e', 'x', 't', '-', 's', 'i', 'z', 'e', '-', 'a', 'd', 'j', 'u', 's', 't',
  };

  uint32_t block;
  memcpy(&block, str + 16, sizeof(block));
  uint32_t pos = uint32_t(block * 0xe330a008U) >> 28;
  const uint8_t *candidate = strings + 26 * pos;
  if (memcmp(candidate + 2, str, 24) == 0) {
    return candidate[0] + (candidate[1] << 8);
  }
  return 0;

You can see that we used a 32-bit value here (bytes 16 through 19 of the input), and a corresponding 32-bit magic (though still not with an AND mask). So we got fairly lucky, but sometimes you do that. Of course, we need to validate the entire 24-byte value even though we only discriminated on four of the bytes! (Unless you know for sure that you never have any out-of-distribution inputs, that is. There are use cases where this is true.)

(If you wonder what 95, 0 or similar is above; that's just “the answer the user wanted for that input”. It corresponds to a 16-bit enum in the parser.)

If there are only a few values, we don't need any of this; just like Wojciech, we do with a simple compare. Here's the generated code for all length-37 CSS keywords, plain and simple:

  if (memcmp(str, "-internal-inactive-list-box-selection", 37) == 0) {
    return 171;
  }
  return 0;

(Again 171 is “the desired output for that input”, not a value the code generator decides in any way.)

So how do we find these magic values? There's really only one way: Try lots of different ones and see if they work. But there's a trick to accelerate “see if they work”, which I also borrowed from computer chess: The killer heuristic.

See, to try if a magic is good, you generally try to hash all the different values and see if any two go into the same bucket. (If they do, it's not a perfect hash and the entire point of the exercise is gone.) But it turns out that most of the time, it's the same two values that collide. So every couple hundred candidates, we check which two values disproved the magic, and put those in a slot. Whenever we check magics, we can now try those first, and more likely than not, discard the candidate right away and move on to the next one (whether it is by exhaustive search or randomness). It's actually a significant speedup.

But occasionally, we simply cannot find a magic for a given group; either there is none, or we didn't have enough time to scan through enough of the 64-bit space. At this point, Wojciech suggests we switch on one of the characters (heuristically) to get smaller subgroups and try again. I didn't actually find this to perform all that well; indirect branch predictors are better than 20 years ago, but the pattern is typically not that predictable. What I tried instead was to have more of a yes/no on some character (i.e., a non-indirect branch), which makes for a coarser split.

It's not at all obvious where the best split would be. You'd intuitively think that 50/50 would be a good idea, but if you have e.g. 40 elements, you'd much rather split them 32/8… if you can find perfect hashes for both subgroups (5-bit and 3-bit, respectively). If not, a 20–20 split is most likely better, since you very easily can find magics that put 20 elements into 32 buckets without collisions. I ended up basically trying all the different splits and scoring them, but this makes the searcher rather slow, and it means you basically must have some sort of cache if you want to run it as part of your build system. This is the part I'm by far the least happy about; gperf isn't great by modern standards, but it never feels slow to run.

The end result for me was: Runtime about twice as fast as gperf, compiled code about half as big. That's with everything hard-coded; if you're pushed for space (or are icache-bound), you could make more generic code at the expense of some speed.

So, if anyone wants to make a more modern gperf, I guess this space is up for grabs? It's not exactly technology that will make your stock go to AI levels, though.

23 October, 2025 08:23PM

hackergotchi for Daniel Pocock

Daniel Pocock

Social engineering attack: Debian voted to trick you on binary blobs

Many people expressed mild dismay about the 2022 vote to allow binary blobs in the Debian installer.

Few people, if any, commented on the trick: they have used the term "non-free" to refer to these blobs.

This is another social engineering attack in broad daylight.

In the Debian world, to classify software as non-free, it still has to provide all of the source code. The term non-free tells us that there is something awkward about the license for that source code but when you see the term non-free, you know the source code is there.

From the Wayback machine snapshot of Debian web site in 1997:

Packages in this directory do not necessarily cost money, but have some onerous condition restricting the redistribution of the software.

Yet if we look at the Debian policy manual today, we find the thin end of the wedge:

It is possible that there are policy requirements which the package is unable to meet, for example, if the source is unavailable. These situations will need to be handled on a case-by-case basis.

The text of the policy manual has been softened up so that somebody with enough power, like an FTP Master, could slip something in under the radar. Remember the concerns expressed by AJ Towns about giving too much power to one person? Does his client exert pressure on him to take shortcuts when necessary to ensure their flights stay on schedule?

Non-free licenses typically have some clauses in them that allow us to redistribute the source code but they make some limit on how the code can be used. For example, the very famous case of the original JSLint license, containing the clause "The Software shall be used for Good, not Evil." is a non-free license. However, the JSLint developers always provided one hundred percent of the source code.

The community has accepted the definition of non-free over three decades. Yet since the vote in 2022, people are using exactly the same term, non-free, to refer to something that is even less free than non-free.

They softened us up to the term non-free over thirty years. The 2022 general resolution vote allows them to use our understanding of the term non-free to exploit us and give us something other than what we came to expect from that term.

IBM Red Hat very explicitly told people that RHEL source code can't be distributed any more. The wording of the Debian general resolution sent us down the same path by stealth.

To put it another way, they voted to broaden the definition of non-free so the term no longer has any value at all.

We saw the same phenomena with the word "harassment". They now use the word "harassment" to describe anything they don't agree with. When competent professionals ask urgent questions about the social engineering attacks, the misfits cry "harassment" to avoid looking us in the eye.

Leaked debian-private archives give us many insights into how the Debian founders put great thought into the meaning of these words.

Remember, they spent over $120,000 on legal fees to argue about the trademark. Why didn't anybody defend the real meaning of the non-free trademark? For many users, the non-free trademark is far more important than nit-picking about pronouns.

Please see the chronological history of how the Debian harassment and abuse culture evolved.

23 October, 2025 05:30AM

Russ Allbery

Review: Politics on the Edge

Review: Politics on the Edge, by Rory Stewart

Publisher: Penguin Books
Copyright: 2023, 2025
Printing: 2025
ISBN: 979-8-217-06167-9
Format: Kindle
Pages: 429

Rory Stewart is a former British diplomat, non-profit executive, member of Parliament, and cabinet minister. Politics on the Edge is a memoir of his time in the UK Parliament from 2019 to 2019 as a Tory (Conservative) representing the Penrith and The Border constituency in northern England. It ends with his failed run against Boris Johnson for leader of the Conservative Party and Prime Minister.

This book provoked many thoughts, only some of which are about the book. You may want to get a beverage; this review will be long.

Since this is a memoir told in chronological order, a timeline may be useful. After Stewart's time as a regional governor in occupied Iraq (see The Prince of the Marshes), he moved to Kabul to found and run an NGO to preserve traditional Afghani arts and buildings (the Turquoise Mountain Foundation, about which I know nothing except what Stewart wrote in this book). By his telling, he found that work deeply rewarding but thought the same politicians who turned Iraq into a mess were going to do the same to Afghanistan. He started looking for ways to influence the politics more directly, which led him first to Harvard and then to stand for Parliament.

The bulk of this book covers Stewart's time as MP for Penrith and The Border. The choice of constituency struck me as symbolic of Stewart's entire career: He was not a resident and had no real connection to the district, which he chose for political reasons and because it was the nearest viable constituency to his actual home in Scotland. But once he decided to run, he moved to the district and seems sincerely earnest in his desire to understand it and become part of its community. After five years as a backbencher, he joined David Cameron's government in a minor role as Minister of State in the Department for Environment, Food, and Rural Affairs. He then bounced through several minor cabinet positions (more on this later) before being elevated to Secretary of State for International Development under Theresa May. When May's government collapsed during the fight over the Brexit agreement, he launched a quixotic challenge to Boris Johnson for leader of the Conservative Party.

I have enjoyed Rory Stewart's writing ever since The Places in Between. This book is no exception. Whatever one's other feelings about Stewart's politics (about which I'll have a great deal more to say), he's a talented memoir writer with an understated and contemplative style and a deft ability to shift from concrete description to philosophical debate without bogging down a story. Politics on the Edge is compelling reading at the prose level. I spent several afternoons happily engrossed in this book and had great difficulty putting it down.

I find Stewart intriguing since, despite being a political conservative, he's neither a neoliberal nor any part of the new right. He is instead an apparently-sincere throwback to a conservatism based on epistemic humility, a veneration of rural life and long-standing traditions, and a deep commitment to the concept of public service. Some of his principles are baffling to me, and I think some of his political views are obvious nonsense, but there were several things that struck me throughout this book that I found admirable and depressingly rare in politics.

First, Stewart seems to learn from his mistakes. This goes beyond admitting when he was wrong and appears to include a willingness to rethink entire philosophical positions based on new experience.

I had entered Iraq supporting the war on the grounds that we could at least produce a better society than Saddam Hussein's. It was one of the greatest mistakes in my life. We attempted to impose programmes made up by Washington think tanks, and reheated in air-conditioned palaces in Baghdad — a new taxation system modelled on Hong Kong; a system of ministers borrowed from Singapore; and free ports, modelled on Dubai. But we did it ultimately at the point of a gun, and our resources, our abstract jargon and optimistic platitudes could not conceal how much Iraqis resented us, how much we were failing, and how humiliating and degrading our work had become. Our mission was a grotesque satire of every liberal aspiration for peace, growth and democracy.

This quote comes from the beginning of this book and is a sentiment Stewart already expressed in The Prince of the Marshes, but he appears to have taken this so seriously that it becomes a theme of his political career. He not only realized how wrong he was on Iraq, he abandoned the entire neoliberal nation-building project without abandoning his belief in the moral obligation of international aid. And he, I think correctly, identified a key source of the error: an ignorant, condescending superiority that dismissed the importance of deep expertise.

Neither they, nor indeed any of the 12,000 peacekeepers and policemen who had been posted to South Sudan from sixty nations, had spent a single night in a rural house, or could complete a sentence in Dinka, Nuer, Azande or Bande. And the international development strategy — written jointly between the donor nations — resembled a fading mission statement found in a new space colony, whose occupants had all been killed in an alien attack.

Second, Stewart sincerely likes ordinary people. This shone through The Places in Between and recurs here in his descriptions of his constituents. He has a profound appreciation for individual people who have spent their life learning some trade or skill, expresses thoughtful and observant appreciation for aspects of local culture, and appears to deeply appreciate time spent around people from wildly different social classes and cultures than his own. Every successful politician can at least fake gregariousness, and perhaps that's all Stewart is doing, but there is something specific and attentive about his descriptions of other people, including long before he decided to enter politics, that makes me think it goes deeper than political savvy.

Third, Stewart has a visceral hatred of incompetence. I think this is the strongest through-line of his politics in this book: Jobs in government are serious, important work; they should be done competently and well; and if one is not capable of doing that, one should not be in government. Stewart himself strikes me as an insecure overachiever: fiercely ambitious, self-critical, a bit of a micromanager (I suspect he would be difficult to work for), but holding himself to high standards and appalled when others do not do the same. This book is scathing towards multiple politicians, particularly Boris Johnson whom Stewart clearly despises, but no one comes off worse than Liz Truss.

David Cameron, I was beginning to realise, had put in charge of environment, food and rural affairs a Secretary of State who openly rejected the idea of rural affairs and who had little interest in landscape, farmers or the environment. I was beginning to wonder whether he could have given her any role she was less suited to — apart perhaps from making her Foreign Secretary. Still, I could also sense why Cameron was mesmerised by her. Her genius lay in exaggerated simplicity. Governing might be about critical thinking; but the new style of politics, of which she was a leading exponent, was not. If critical thinking required humility, this politics demanded absolute confidence: in place of reality, it offered untethered hope; instead of accuracy, vagueness. While critical thinking required scepticism, open-mindedness and an instinct for complexity, the new politics demanded loyalty, partisanship and slogans: not truth and reason but power and manipulation. If Liz Truss worried about the consequences of any of this for the way that government would work, she didn't reveal it.

And finally, Stewart has a deeply-held belief in state capacity and capability. He and I may disagree on the appropriate size and role of the government in society, but no one would be more disgusted by an intentional project to cripple government in order to shrink it than Stewart.

One of his most-repeated criticisms of the UK political system in this book is the way the cabinet is formed. All ministers and secretaries come from members of Parliament and therefore branches of government are led by people with no relevant expertise. This is made worse by constant cabinet reshuffles that invalidate whatever small amounts of knowledge a minister was able to gain in nine months or a year in post. The center portion of this book records Stewart's time being shuffled from rural affairs to international development to Africa to prisons, with each move representing a complete reset of the political office and no transfer of knowledge whatsoever.

A month earlier, they had been anticipating every nuance of Minister Rogerson's diary, supporting him on shifts twenty-four hours a day, seven days a week. But it was already clear that there would be no pretence of a handover — no explanation of my predecessor's strategy, and uncompleted initiatives. The arrival of a new minister was Groundhog Day. Dan Rogerson was not a ghost haunting my office, he was an absence, whose former existence was suggested only by the black plastic comb.

After each reshuffle, Stewart writes of trying to absorb briefings, do research, and learn enough about his new responsibilities to have the hope of making good decisions, while growing increasingly frustrated with the system and the lack of interest by most of his colleagues in doing the same. He wants government programs to be successful and believes success requires expertise and careful management by the politicians, not only by the civil servants, a position that to me both feels obviously correct and entirely at odds with politics as currently practiced.

I found this a fascinating book to read during the accelerating collapse of neoliberalism in the US and, to judge by current polling results, the UK. I have a theory that the political press are so devoted to a simplistic left-right political axis based on seating arrangements during the French Revolution that they are missing a significant minority whose primary political motivation is contempt for arrogant incompetence. They could be convinced to vote for Sanders or Trump, for Polanski or Farage, but will never vote for Biden, Starmer, Romney, or Sunak.

Such voters are incomprehensible to those who closely follow and debate policies because their hostile reaction to the center is not about policies. It's about lack of trust and a nebulous desire for justice. They've been promised technocratic competence and the invisible hand of market forces for most of their lives, and all of it looks like lies. Everyday living is more precarious, more frustrating, more abusive and dehumanizing, and more anxious, despite (or because of) this wholehearted embrace of economic "freedom." They're sick of every complaint about the increasing difficulty of life being met with accusations about their ability and work ethic, and of being forced to endure another round of austerity by people who then catch a helicopter ride to a party on some billionaire's yacht.

Some of this is inherent in the deep structural weaknesses in neoliberal ideology, but this is worse than an ideological failure. The degree to which neoliberalism started as a project of sincere political thinkers is arguable, but that is clearly not true today. The elite class in politics and business is now thoroughly captured by people whose primary skill is the marginal manipulation of complex systems for their own power and benefit. They are less libertarian ideologues than narcissistic mediocrities. We are governed by management consultants. They are firmly convinced their organizational expertise is universal, and consider the specific business of the company, or government department, irrelevant.

Given that context, I found Stewart's instinctive revulsion towards David Cameron quite revealing. Stewart, later in the book, tries to give Cameron some credit by citing several policy accomplishments and comparing him favorably to Boris Johnson (which, true, is a bar Cameron probably flops over). But I think Stewart's baffled astonishment at Cameron's vapidity says a great deal about how we have ended up where we are. This last quote is long, but I think it provides a good feel for Stewart's argument in this book.

But Cameron, who was rumoured to be sceptical about nation-building projects, only nodded, and then looking confidently up and down the table said, "Well, at least we all agree on one extremely straightforward and simple point, which is that our troops are doing very difficult and important work and we should all support them."

It was an odd statement to make to civilians running humanitarian operations on the ground. I felt I should speak. "No, with respect, we do not agree with that. Insofar as we have focused on the troops, we have just been explaining that what the troops are doing is often futile, and in many cases making things worse." Two small red dots appeared on his cheeks. Then his face formed back into a smile. He thanked us, told us he was out of time, shook all our hands, and left the room.

Later, I saw him repeat the same line in interviews: "the purpose of this visit is straightforward... it is to show support for what our troops are doing in Afghanistan". The line had been written, in London, I assumed, and tested on focus groups. But he wanted to convince himself it was also a position of principle.

"David has decided," one of his aides explained, when I met him later, "that one cannot criticise a war when there are troops on the ground."

"Why?"

"Well... we have had that debate. But he feels it is a principle of British government."

"But Churchill criticised the conduct of the Boer War; Pitt the war with America. Why can't he criticise wars?"

"British soldiers are losing their lives in this war, and we can't suggest they have died in vain."

"But more will die, if no one speaks up..."

"It is a principle thing. And he has made his decision. For him and the party."

"Does this apply to Iraq too?"

"Yes. Again he understands what you are saying, but he voted to support the Iraq War, and troops are on the ground."

"But surely he can say he's changed his mind?"

The aide didn't answer, but instead concentrated on his food. "It is so difficult," he resumed, "to get any coverage of our trip." He paused again. "If David writes a column about Afghanistan, we will struggle to get it published."

"But what would he say in an article anyway?" I asked.

"We can talk about that later. But how do you get your articles on Afghanistan published?"

I remembered how the US politicians and officials had shown their mastery of strategy and detail. I remembered the earnestness of Gordon Brown when I had briefed him on Iraq. Cameron seemed somehow less serious. I wrote as much in a column in the New York Times, saying that I was afraid the party of Churchill was becoming the party of Bertie Wooster.

I don't know Stewart's reputation in Britain, or in the constituency that he represented. I know he's been accused of being a self-aggrandizing publicity hound, and to some extent this is probably true. It's hard to find an ambitious politician who does not have that instinct. But whatever Stewart's flaws, he can, at least, defend his politics with more substance than a corporate motto. One gets the impression that he would respond favorably to demonstrated competence linked to a careful argument, even if he disagreed. Perhaps this is an illusion created by his writing, but even if so, it's a step in the right direction.

When people become angry enough at a failing status quo, any option that promises radical change and punishment for the current incompetents will sound appealing. The default collapse is towards demagogues who are skilled at expressing anger and disgust and are willing to promise simple cures because they are indifferent to honesty. Much of the political establishment in the US, and possibly (to the small degree that I can analyze it from an occasional news article) in the UK, can identify the peril of the demagogue, but they have no solution other than a return to "politics as usual," represented by the amoral mediocrity of a McKinsey consultant. The rare politicians who seem to believe in something, who will argue for personal expertise and humility, who are disgusted by incompetence and have no patience for facile platitudes, are a breath of fresh air.

There are a lot of policies on which Stewart and I would disagree, and perhaps some of his apparent humility is an affectation from the rhetorical world of the 1800s that he clearly wishes he were inhabiting, but he gives the strong impression of someone who would shoulder a responsibility and attempt to execute it with competence and attention to detail. He views government as a job, where coworkers should cooperate to achieve defined goals, rather than a reality TV show. The arc of this book, like the arc of current politics, is the victory of the reality TV show over the workplace, and the story of Stewart's run against Boris Johnson is hard reading because of it, but there's a portrayal here of a different attitude towards politics that I found deeply rewarding.

If you liked Stewart's previous work, or if you want an inside look at parliamentary politics, highly recommended. I will be thinking about this book for a long time.

Rating: 9 out of 10

23 October, 2025 04:47AM

October 22, 2025

hackergotchi for Daniel Pocock

Daniel Pocock

Mysterious grant forfeited, $100,000 from Software in the Public Interest accounts 2023

Techrights has recently reported on a grant of $100,000 forfeited by Software in the Public Interest, Inc (SPI).

SPI acts as an umbrella for multiple free and open source software projects. Their form 990 public filings do not give a per-project breakdown.

Somebody may be able to pick through the month-by-month financial reports and identify when they received this grant, when they gave it back and which organization it relates to. Or somebody could try to ask them.

Grants often come with conditions. In some cases, an organization applies for a grant in good faith and later on, after receiving the money, they decide that spending the money, under the conditions they agreed to, is no longer profitable for the organization so they give it back.

Sometimes the conditions include specific targets or deadlines. In those cases, the organization may be doing some work but they miss the deadline. If they incurred expenses related to the project and then gave the money back then the community needs to look very closely at what went wrong.

During 2025, the new US Government administration has canceled many grants. The reports being discussed from SPI relate to the return of a grant in 2023, prior to the Trump administration.

Prior to the death of Abraham Raji, the accounts from SPI showed us over $120,000 spent on legal fees for censoring discussions about Debianism. Yet DebConf organizers asked volunteers to contribute their own money to the day trip and Abraham Raji was left alone, swimming unsupervised, and he drowned.

While waiting for more details about this money from 2023, now is a good time to go and read through the leaked debian-private archives to see some of the discussions about the secret strategy for SPI and why SPI was created separately from Debianism.

Read more about governance issues at Software in the Public Interest, Inc.

22 October, 2025 01:00PM

Evidence: bullying, student union behaviour: Armijn Hemel’s FSFE resignation

A few years ago, Armijn Hemel silently resigned from the FSFE misfits.

But it wasn't really a silent resignation. He sent an internal resignation email citing the behaviour of certain bullies and telling people that their behaviour reminds him of a student union.

* Bad organizational management

FSFE feels like it is reverting back to being run like a student union, whereas in the past people tried to make it more professional. FSFE's management makes very basic and unnecessary mistakes unworthy of an organization trying to have global influence. This is hurting FSFE. Even though this has been addressed a few times (by some, including myself) nothing seems to be done to rectify these mistakes.

and this really jumps out at us:

* No improvements/personal attacks

None of the issues mentioned above are new (and if they are, then QED) and have been mentioned before. What I have noticed when issues are mentioned is that either one of these three things happens:

- they fall on deaf ears

- they are acknowledged and then ignored

- FSFE gets very defensive, or launches personal attacks. The most glaring case was when the vice-president recently came barging in and arrogantly threw insults left and right without understanding any of the background of the situation, and launched a vicious personal attack on two of the previous legal coordinators.

and Armijn's conclusion:

All these things combined have tipped the scales and now the negative far outweighs the positive (because there is very little positive left in my opinion). I no longer feel that FSFE is the right place for me, or that it is advancing free software in Europe (especially the legal field) and it is better to focus my energy somewhere else where I can have a more positive impact.

Read more about the FSFE misfits usurping the name of Dr Richard Stallman's FSF.

22 October, 2025 12:00PM

Evidence: psychological abuse, stalking, Galia Mancheva, Susanne Eiswirt ignored by FSFE judgment for Matthias Kirschner

In 2020, we know that Susanne Eiswirt suddenly left the FSFE misfits and Galia Mancheva was forced to take long term medical leave.

Mancheva started a law suit against the FSFE misfits and Matthias Kirschner in particular.

In Germany, women seem to have very few rights and the court gave a judgment in favour of Matthias Kirschner.

Nonetheless, Matthias Kirschner did not deny the allegation that he went to the home of a female employee even after being given written notice not to do so ever again.

Copied from the blog of Galia Mancheva:

————– I took FSFE to court. This is my story ––––– Soon after the first lockdown in Berlin this year I filed a public case in the Berlin Tribunal of Labour Court against the president of Free Software Foundation Europe (FSFE), Matthias Kirschner, for workplace bullying. Why? A female colleague and me had dared to discuss wage transparency and gender pay gap in the office. Apparently it is common in Germany that this gap exceeds 20%, but we both felt secure that the free software movement is progressive, and cares about being inclusive and equal opportunities oriented. Unfortunately we miscalculated – our boss Matthias was beyond furious. After that office meeting, he told my colleague “there will be consequences”. Our efforts coincided with the resignation of Richard Stallman from the US-based sister organisation of FSFE due to careless revictimisation of female victims of sexual abuse- another gender discrimination issue in our community that would cause the situation in our office to deteriorate quickly. In its reluctant press release on this pivotal change in leadership in the largest free software organisation in the world, the FSFE had opted to honour Stallman for his undeniably long service and overlook the social issues underlying the change – something with which I expressed dissatisfaction, and not without support from colleagues. It led to immediate retribution. I was ordered to rewrite the text and was warned that I had “three hours to do it. Whether we will publish it or not, is going to be my [Matthias', my rem.] decision, not yours”. Free software is in most of our digital infrastructure, and I care a lot about inclusivity in this community to ensure that our most basic tools can be developed by everyone's perspectives for everyone's needs, so I rewrote our announcement. But not only was it never published – it was not even honoured with his feedback. My aforementioned female colleague, who had also backed me up, was fired just a few days later. Personally, I was subjected to a good amount of pressure and strategic intrigues. I was given tasks irrelevant to my job description and far below my qualifications. I was degraded both as a professional and due to who I am through instructions such as “translate this [text] in your mother tongue, so you can understand better”. I was belligerently micromanaged and questioned through rebukes like “why do you refer to dates in this format and not in that format”, while there is no office policy on following a particular date standard. I was even told some of the things I was working on were bullshit! For 3 months I was pushing for time to launch a newsletter survey as an attempt to make qualitative improvements, but my effort was labelled as “not necessary”. I was regularly nagged, and prevented from doing the work I was hired for. In front of my colleagues I was tasked with one thing and in private I was asked not to do it and have it replaced with another, and then in front of others asked about the progress on the previous. Matthias was manufacturing the false impression I wasn't doing my work, while at the same time calling and texting my private phone number in obscure hours (such as 5:00AM or 22:00PM) with work orders and topics. Unlike other colleagues, I had a hard time receiving time off for the extra days I worked on, and was prevented from taking any of my annual holidays. I suspected Matthias was preparing to fire me, and indeed I was on a clock. He needed to make sure he was re-elected FSFE president before he could get rid of me. It would have damaged his chances to have fired all the full-time women in the office in the two months leading up to the elections – an unnecessary risk – but once approved he would have another two years' free reign. At this time my friends were starting to worry – the psychological pressure and lack of enough time off caused my condition was decline. I had to take a sick leave. Certain that I was about to get fired, my friends encouraged me to seek legal counsel, if so only to keep myself focussed on constructive tasks. Immediately after my sick leave announcement, Matthias fired me over the phone on a Friday night, and threatened me to immediately go to the office to hand over work-related items. Appearing in the office during a sick leave is illegal in Germany, so I refused. A weekend of non-stop calls followed, including from hidden phone numbers. He even texted telling me I should answer my phone, for my own better. Even after my lawyer warned him to terminate all attempts to communicate with me and send someone else to pick up my work laptop, he came in person to my house, and was very irritated that I was not alone. Eventually, of course, my sick leave ended, I was fired, and there was a global pandemic with follow-on lockdowns. It was against this background that I filed a complaint in the Anti-Discrimination Commission in Germany and I filed a case against him in court for work-place bullying. Disappointingly, it turned out, the Commission does not have much legal authority for actions, so I proceeded further only with the bullying case. During the process, one curious clue popped up. In one of the publicly available answers to the court he argued I had no reason to doubt a gender pay gap, because you see, there was this male trainee that was having a lower pay than me. But, not only does the FSFE rate women salaries lower, but also foreigners. So, if you happen to be a foreign woman – well, tough luck. At the same time they are proudly proclaiming the office “international” and “inclusive” when pursuing donor relations or in relations to the public. The court process was taxing. The FSFE lawyer made up easily disprovable slanders against me – and I say “easily” because their charges were demonstrably false, but of course finding evidence that is also admissible in the conservative German court system did not do much good for my stress levels over summer. They disrespected the court by not submitting papers on time, and they refused to answer my allegations, opting instead to portray me as a disobedient, sexist, racist, incompetent belligerent. Why did they give such a person a permanent contract after a six month probation period? Well, magnanimously they apparently were afraid that I would otherwise be disappointed. Now, I finally received the Labour court's verdict some days ago. The court basically says that even though they recognise my claims as true I do not qualify for financial remuneration because I did not suffer for at least a year and I did not end up having major psychological damage of my identity. I was accused by the FSFE that my claim was driven by a desire for a quick financial gain. Unfortunately, the German law foresees only monetary and no other type of compensation. However, I was given the opportunity in court to ask for something else. I asked for an apology and the president of the FSFE refused. Why am I writing all this? Because I want to expose the hypocrisy and double standards in FSFE leadership. How the organisation “promoting” transparency, equality and inclusivity treats employees and more specifically women. How donations are spent on nurturing Matthias' grandiosity. Because, let's be honest, to how many male employees' homes would he appear uninvited after being cautioned by a lawyer not to do so? I remain committed to open infrastructure and free software, and I know the community extends beyond the FSFE Berlin office, where Matthias reigns supreme with the support of his FSFE Board employees. I of course also hope that the advocacy arm of the European free software movement will eventually also reflect the diversity, friendship and equality that I continue to find in the movement as such. Finally, I want to thank those of my hacker friends who showed up in person at my final court hearing and those whose priceless support remained in spite of physical distance. Your presence made it so much more bearable, and reminded me that it is not the sour apples of the FSFE offices that make our community, but the many people around them who continue to commit themselves and their skills to a better tomorrow.
Susanne Eiswirt, Matthias Kirschner, Galia Mancheva, FSFE, workplace bullying, harassment

 

Read more about the FSFE misfits usurping the name of Dr Richard Stallman's FSF.

22 October, 2025 10:00AM

Helping FSFE scam victims and conference organisers

From time to time I receive emails from people who saw one of the fake FSFE misfits listed as a speaker at a conference. Sometimes the name of the fake FSFE is mentioned in the conference program.

From time to time, we see conference organizers using the name FSF and FSFE interchangably. They simply don't realize that the fake FSFE misfits are a totally different group of people. The real FSF and the real Dr Richard Stallman have no management or control over the activities of the fake FSFE misfits. Yet third parties don't realize that and sometimes we see people using FSF and FSFE in the same sentence.

When people realize they have been tricked, there is always a feeling of embarassment and sometimes a feeling of shame. Some people don't like to admit they've been tricked.

In many cases, a conference organizer will consider making small changes to their web site to remove references to the FSF and FSFE. They can make these changes without having to lose face, without having to admit they were tricked and without creating a conflict situation between their event and the rogue FSFE misfits.

Nonetheless, when somebody from the fake FSFE misfits gets up to speak at a conference, they inevitably demonstrate links to the FSFE web site and when they have a table or booth at a conference, they distribute printed materials with the FSFE web address on them. It is vital to limit the extent to which the community is exposed to these materials.

Therefore, the approach I recommend is to be kind and patient with conference organizers while also showing them the strongest evidence to help them make a decision. Help them to avoid losing face but also help them understand the consequences.

The strongest evidence, by far, is the story of the volunteer who gave a bequest of EUR 150,00 to the fake FSFE misfits. It is important to emphasize to conference organizers that somebody in their audience could be tricked and make the same mistake. Once people understand this, they have a legitimate reason to totally remove fake FSFE misfits from the conference agenda.

The second strongest evidence are the leaked emails revealing that FSF has complained about FSFE impersonating them. As these emails come from the two organizations concerned, they are definitive proof that the name FSFE is not valid.

Therefore, when communicating with a conference organizer, you could use this approach:

I was looking at the schedule and I saw that Matthias Kirschner is a speaker. He is listed as a representative of the FSFE.

In 2018, the community representative discovered that a volunteer had made a bequest of €150,000 to the FSFE when he died. The representative was concerned that the donor had been tricked and may have intended to give this money to the real FSF in Boston.

The FSF and FSFE are now two completely different groups with different presidents. The leaked internal emails show us that the real FSF made a complaint about the name of the FSFE. The FSFE managers, both the president and executive director, made various written comments admitting they have no grounds to use the name FSFE. The FSFE president admitted that people have been confused.

Please contact the real FSF and ask them if the speaker has permission to represent FSF and Dr Richard Stallman. If the FSF does not give permission, I feel the speaker should be removed.

The leaked resignation of Armijn Hemel is one of the most compelling documents in the list of all the FSFE resignations.

A female employee of the FSFE, Galia Mancheva, went to court against Matthias Kirschner seeking a judgment for unfair dismissal. In her public testimony, she alleges coercive control, psychological abuse and stalking at her home. Please read it.

I feel a strong concern that participants in your event may give other speaking opportunities to the same speaker in future. Some people may give donations or voluntary work to the wrong FSFE. Therefore, while I realize it is not easy to cancel a speech, the risk of future consequences is big enough to justify the immediate loss of face.

The speaker may be able to obtain endorsement from the real FSF or change their affiliation to remove references to the FSFE. But if the conference proceeds as it is now, people will definitely be confused by the name FSFE.

Read more about the FSFE misfits usurping the name of Dr Richard Stallman's FSF.

22 October, 2025 08:00AM

October 21, 2025

hackergotchi for Gunnar Wolf

Gunnar Wolf

LLM Hallucinations in Practical Code Generation — Phenomena, Mechanism, and Mitigation

This post is an unpublished review for LLM Hallucinations in Practical Code Generation — Phenomena, Mechanism, and Mitigation

How good can Large Language Models (LLMs) be at generating code? This would not seem like a very novel question to ask, as there are several benchmarks such as HumanEval and MBPP, published in 2021, before LLMs burst into the public view starting the current AI inflation. However, this article’s authors point out code generation is very seldom done as isolated functions, but must be deployed in a coherent fashion together with the rest of the project or repository it is meant to be integrated in. By 2024 there are several benchmarks, such as CoderEval o EvoCodeBench, measuring functional correctness of LLM-generated code, measured by test case pass rates.

This article brings a new proposal to the table: comparing LLM-generated repository-level-evaluated code, by examining the hallucinations generated. To do this, they begin by running the Python code generation tasks proposed in the CoderEval benchmark against six code-generating LLMs, analizing the results and building a taxonomy to describe code-based LLM hallucinations, with three types of conflicts (task requirement, factual knowledge and project context) as first-level categories and eight subcategories within them. Second, the authors compare the results of each of the LLMs per main hallucination category. Third, they try to find the root cause for the hallucinations.

The article is structured very clearly, not only presenting the three Research Questions (RQ) but also refering to them as needed to explain why and how each partial result is interpreted. RQ1 (establishing a hallucination taxonomy) is, in my opinion, the most thoroughly explored. RQ2 (LLM comparison) is clear, although it just presents “straight” results, seemingly without much analysis to them. RQ3 (root cause discussion) is undoubtedly interesting, but I feel it to be much more speculative and not directly related to the analysis performed.

After tackling their research questions, they venture a possible mitigation to counter the effect of hallucinations: enhancing the presented LLMs with a retrieval-augmented generation (RAG), so it better understands task requirements, factual knowledge and project context, hopefully reducing hallucination; they present results showing all of the models are clearly although modestly improved by the proposed RAG-based mitigation.

The article is clearly written and easy to read. I would like they would have dedicated more space to detail their RAG implementation, but I suppose it will appear in a follow-up article, as it was only briefly mentioned here. It should provide its target audience, is quite specialized but numerous nowadays, with interesting insights and discussion.

21 October, 2025 10:08PM

Russ Allbery

Review: Space Trucker Jess

Review: Space Trucker Jess, by Matthew Kressel

Publisher: Fairwood Press
Copyright: July 2025
ISBN: 1-958880-27-2
Format: Kindle
Pages: 472

Space Trucker Jess is a stand-alone far-future space fantasy novel.

Jess is a sixteen-year-old mechanic working grey-market jobs on Chadeisson Station with a couple of younger kids. She's there because her charming and utterly unreliable father got caught running a crypto scam and is sitting in detention. This was only the latest in a long series of scams, con jobs, and misadventures she's been dragged through since her mother disappeared without a word. Jess is cynical, world-weary, and infuriated by her own sputtering loyalty to her good-for-nothing dad.

What Jess wants most in the universe is to own a CCM 6454 Spark Megahauler, the absolute best cargo ship in the universe according to Jess. She should know; she's worked on nearly every type of ship in existence. With her own ship, she could make a living hauling cargo, repairing her own ship, and going anywhere she wants, free of her father and his endless schemes. (A romantic relationship with her friend Leurie would be a nice bonus.)

Then her father is taken off the station on a ship leaving the galactic plane, no one will tell her why, and all the records of the ship appear to have been erased.

Jess thinks her father is an asshole, but that doesn't mean she can sit idly by when he disappears. That's how she ends up getting in serious trouble with station security due to some risky in-person sleuthing, followed by an expensive flight off the station with a dodgy guy and a kid in a stolen spaceship.

The setup for this book was so great. Kressel felt the need to make up a futuristic slang for Jess and her friends to speak, which rarely works as well as the author expects and does not work here, but apart from that I was hooked. Jess is sarcastic, blustery, and a bit of a con artist herself, but with the idealistic sincerity of someone who knows that her life is been kind of broken and understands the value of friends. She's profoundly cynical in the heartbreakingly defensive way of a sixteen-year-old with a rough life. I have a soft spot in my heart for working-class science fiction (there isn't nearly enough of it), and there are few things I enjoy more than reading about the kind of protagonist who has Opinions about starship models and a dislike of shoddy work. I think this is the only book I've bought solely on the basis of one of the Big Idea blog posts John Scalzi hosts.

I really wish this book had stuck with the setup instead of morphing into a weird drug-enabled mystical space fantasy, to which Jess's family is bizarrely central.

SPOILERS below because I can't figure out how to rant about what annoyed me without them. Search for the next occurrence of spoilers to skip past them.

There are three places where this book lost me. The first was when Jess, after agreeing to help another kid find his father, ends up on a world obsessed with a religious cult involving using hallucinatory drugs to commune with alien gods. Jess immediately flags this as unbelievable bullshit and I was enjoying her well-founded cynicism until Kressel pulls the rug out from under both Jess and the reader by establishing that this new-age claptrap is essentially true.

Kressel does try to put a bit of a science fiction gloss on it, but sadly I think that effort was unsuccessful. Sometimes absurdly powerful advanced aliens with near-telepathic powers are part of the fun of a good space opera, but I want the author to make an effort to connect the aliens to plausibility or, failing that, at least avoid sounding indistinguishable from psychic self-help grifters or religious fantasy about spiritual warfare. Stargate SG-1 and Babylon 5 failed on the first part but at least held the second line. Kressel gets depressingly close to Seth territory, although at least Jess is allowed to retain some cynicism about motives.

The second, related problem is that Jess ends up being a sort of Chosen One, which I found intensely annoying. This may be a fault of reader expectations more than authorial skill, but one of the things I like to see in working-class science fiction is for the protagonist to not be absurdly central to the future of the galaxy, or to at least force themselves into that position through their own ethics and hard work. This book turns into a sort of quest story with epic fantasy stakes, which I thought was much less interesting than the story the start of the book promised and which made Jess a less interesting character.

Finally, this is one of those books where Jess's family troubles and the plot she stumbles across turn into the same plot. Space Trucker Jess is far from alone in having that plot structure, and that's the problem. I'm not universally opposed to this story shape, but Jess felt like the wrong character for it. She starts the story with a lot of self-awareness about how messed up her family dynamics were, and I was rooting for her to find some space to construct her own identity separate from her family. To have her family turn out to be central not only to this story but to the entire galaxy felt like it undermined that human core of the story, although I admit it's a good analogy to the type of drama escalation that dysfunctional families throw at anyone attempting to separate from them.

Spoilers end here.

I rather enjoyed the first third of this book, despite being a bit annoyed at the constructed slang, and then started rolling my eyes and muttering things about the story going off the rails. Jess is a compelling enough character (and I'm stubborn enough) that I did finish the book, so I can say that I liked the very end. Kressel does finally arrive at the sort of story that I wanted to read all along. Unfortunately, I didn't enjoy the path he took to get there.

I think much of my problem was that I wanted Jess to be a more defiant character earlier in the novel, and I wanted her family problems to influence her character growth but not be central to her story. Both of these may be matters of opinion and an artifact of coming into the book with the wrong assumptions. If you are interested in a flawed and backsliding effort to untangle one's identity from a dysfunctional family and don't mind some barely-SF space mysticism and chosen one vibes, it's possible this book will click with you. It's not one that I can recommend, though.

I still want the book that I hoped I was getting from that Big Idea piece.

Rating: 4 out of 10

21 October, 2025 03:36AM

October 20, 2025

hackergotchi for Matthew Garrett

Matthew Garrett

Where are we on X Chat security?

AWS had an outage today and Signal was unavailable for some users for a while. This has confused some people, including Elon Musk, who are concerned that having a dependency on AWS means that Signal could somehow be compromised by anyone with sufficient influence over AWS (it can't). Which means we're back to the richest man in the world recommending his own "X Chat", saying The messages are fully encrypted with no advertising hooks or strange “AWS dependencies” such that I can’t read your messages even if someone put a gun to my head.

Elon is either uninformed about his own product, lying, or both.

As I wrote back in June, X Chat genuinely end-to-end encrypted, but ownership of the keys is complicated. The encryption key is stored using the Juicebox protocol, sharded between multiple backends. Two of these are asserted to be HSM backed - a discussion of the commissioning ceremony was recently posted here. I have not watched the almost 7 hours of video to verify that this was performed correctly, and I also haven't been able to verify that the public keys included in the post were the keys generated during the ceremony, although that may be down to me just not finding the appropriate point in the video (sorry, Twitter's video hosting doesn't appear to have any skip feature and would frequently just sit spinning if I tried to seek to far and I should probably just download them and figure it out but I'm not doing that now). With enough effort it would probably also have been possible to fake the entire thing - I have no reason to believe that this has happened, but it's not externally verifiable.

But let's assume these published public keys are legitimately the ones used in the HSM Juicebox realms[1] and that everything was done correctly. Does that prevent Elon from obtaining your key and decrypting your messages? No.

On startup, the X Chat client makes an API call called GetPublicKeysResult, and the public keys of the realms are returned. Right now when I make that call I get the public keys listed above, so there's at least some indication that I'm going to be communicating with actual HSMs. But what if that API call returned different keys? Could Elon stick a proxy in front of the HSMs and grab a cleartext portion of the key shards? Yes, he absolutely could, and then he'd be able to decrypt your messages.

(I will accept that there is a plausible argument that Elon is telling the truth in that even if you held a gun to his head he's not smart enough to be able to do this himself, but that'd be true even if there were no security whatsoever, so it still says nothing about the security of his product)

The solution to this is remote attestation - a process where the device you're speaking to proves its identity to you. In theory the endpoint could attest that it's an HSM running this specific code, and we could look at the Juicebox repo and verify that it's that code and hasn't been tampered with, and then we'd know that our communication channel was secure. Elon hasn't done that, despite it being table stakes for this sort of thing (Signal uses remote attestation to verify the enclave code used for private contact discovery, for instance, which ensures that the client will refuse to hand over any data until it's verified the identity and state of the enclave). There's no excuse whatsoever to build a new end-to-end encrypted messenger which relies on a network service for security without providing a trustworthy mechanism to verify you're speaking to the real service.

We know how to do this properly. We have done for years. Launching without it is unforgivable.

[1] There are three Juicebox realms overall, one of which doesn't appear to use HSMs, but you need at least two in order to obtain the key so at least part of the key will always be held in HSMs

comment count unavailable comments

20 October, 2025 11:36PM

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

RcppArmadillo 15.2.0-0 on GitHub: New Upstream, Simpler OpenMP

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1270 other packages on CRAN, downloaded 42 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 650 times according to Google Scholar.

This versions updates to the 15.2.0 upstream release made today. It brings a few changes over Armadillo 15.0 (see below for more). It follows the most recent RcppArmadillo 15.0.2-2 release and the Armadillo 15 upstream transition with its dual focus on moving on from C++11 and deprecation of a number of API access points. As we had a few releases last month to manage the transition, we will sit this upgrade out and not upload to CRAN in order to normalize our update cadence towards the desired ‘about six in six months’ (that the CRAN Policy asks for). One can of course install as usual directly from the GitHub repository as well as from r-universe which also offers binaries for all CRAN platforms.

The transition to Armadillo 15 appears to be going slowly but steadily. We had well over 300 packages with either a need to relax the C++11 setting and/or update away from now-deprecated API access points. That number has been cut in half thanks to a lot of work from a lot of package maintainers—which is really appreciated! Of course, a lot remains to be done. Issues #489 and #491 contain the over sixty PRs and patches I prepared for all packages with at least one reverse dependency. Most (but not all) have aided in CRAN updates, some packages are still outstanding in terms of updates. As before meta-issue #475 regroups all the resources for the transition. If you, dear reader, have a package that is affected and I could be of assistance please do reach out.

The other change we made is to greatly simplify the detection and setup of OpenMP. As before, we rely on configure to attempt compilation of a minimal OpenMP-using program in order to pass the ‘success or failure’ onto Armadillo as a ‘can-or-cannot’ use OpenMP. In the year 2025 one of the leading consumer brands still cannot ship an OS where this works out of the box, so we try to aide there. For all others systems, R actually covers this pretty well and has a reliable configuration variable that we rely upon. Just as we recommend for downstream users of the package. This setup should be robust, but is a change so by all means if you knowingly rely on OpenMP please test and report back.

The detailed changes since the last CRAN release follow.

Changes in RcppArmadillo version 15.2.0-0 (2025-10-20) (GitHub Only)

  • Upgraded to Armadillo release 15.2.0 (Medium Roast Deluxe)

    • Added rande() for generating matrices with elements from exponential distributions

    • shift() has been deprecated in favour of circshift(), for consistency with Matlab/Octave

    • Reworked detection of aliasing, leading to more efficient compiled code

  • OpenMP detection in configure has been simplified

More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

20 October, 2025 09:13PM

hackergotchi for Thomas Lange

Thomas Lange

New FAI images available, Rocky Linux 10 and AlmaLinux 10 support

New FAI ISOs using FAI 6.4.3 are available. They are using Debian 13 aka trixie, kernel 6.12 and you can now install Rocky Linux 10 and AlmaLinux 10 using these images.

There's also a variant for installing Linux Mint 22.2 and Ubuntu 24.04 which includes all packages on the ISO.

20 October, 2025 08:18PM

Birger Schacht

A plea for

A couple of weeks ago there was an article on the Freexian blog about Using JavaScript in Debusine without depending on JavaScript. It describes how JavaScript is used in the Debusine Django app, namely “for progressive enhancement rather than core functionality”. This is an approach I also follow when implementing web interfaces and I think developments in web technologies and standardization in recent years have made this a lot easier.

One of the examples described in the post, the “Bootstrap toast” messages, was something that I implemented myself recently, in a similar but slightly different way.

In the main app I develop for my day job we also use the Bootstrap framework. I have also used it for different personal projects (for example the GSOC project I did for Debian in 2018, was also a Django app that used Bootstrap). Bootstrap is still primarily a CSS framework, but it also comes with a JavaScript library for some functionality. Previous versions of Bootstrap depended on jQuery, but since version 5 of Bootstrap, you don’t need jQuery anymore. In my experience, two of the more commonly used JavaScript utilities of Bootstrap are modals (also called lightbox or popup, they are elements that are displayed “above” the main content of a website) and toasts (also called alerts, they are little notification windows that often disappear after a timeout). The thing is, Bootstrap 5 was released in 2021 and a lot has happened since then regarding web technologies. I believe that both these UI components can nowadays be implemented using standard HTML5 elements.

An eye opening talk I watched was Stop using JS for that from last years JSConf(!). In this talk the speaker argues that the Rule of least power is one of the core principles of web development, which means we should use HTML over CSS and CSS over JavaScript. And the speaker also presents some CSS rules and HTML elements that added recently and that help to make that happen, one of them being the dialog element:

The <dialog> HTML element represents a modal or non-modal dialog box or other interactive component, such as a dismissible alert, inspector, or subwindow.

The Dialog element at MDN

The baseline for this element is “widely available”:

This feature is well established and works across many devices and browser versions. It’s been available across browsers since March 2022.

The Dialog element at MDN

This means there is an HTML element that does what a modal Bootstrap does!

Once I had watched that talk I removed all my Bootstrap modals and replaced them with HTML <dialog> elements (JavaScript is still needed to .show() and .close() the elements, though, but those are two methods instead of a full library). This meant not only that I replaced code that depended on an external library, I’m now also a lot more flexible regarding the styling of the elements.

When I started implementing notifications for our app, my first approach was to use Bootstrap toasts, similar to how it is implemented in Debusine. But looking at the amount of HTML code I had to write for a simple toast message, I thought that it might be possible to also implement toasts with the <dialog> element. I mean, basically it is the same, only the styling is a bit different. So what I did was that I added a #snackbar area to the DOM of the app. This would be the container for the toast messages. All the toast messages are simply <dialog> elements with the open attribute, which means that they are visible right away when the page loads.

<div id="snackbar">
  {% for message in messages %}
    <dialog class="mytoast alert alert-{{ message.tags }}" role="alert" open>
      {{ message }}
    </dialog>
  {% endfor %}
</div>

This looks a lot simpler than the Bootstrap toasts would have.

To make the <dialog> elements a little bit more fancy, I added some CSS to make them fade in and out:

.mytoast {
    z-index: 1;
    animation: fadein 0.5s, fadeout 0.5s 2.6s;
}

@keyframes fadein {
    from {
        opacity: 0;
    }

    to {
        opacity: 1;
    }
}

@keyframes fadeout {
    from {
        opacity: 1;
    }

    to {
        opacity: 0;
    }
}

To close a <dialog> element once it has faded away, I had to add one JavaScript event listener:

window.addEventListener('load', () => {
    document.querySelectorAll(".mytoast").forEach((element) => {
        element.addEventListener('animationend', function(e) {
            e.animationName == "fadeout" && element.close();
        });
    });
});

(If one would want to use the same HTML code for both script and noscript users, then the CSS should probably adapted: it fades away and if there is no JavaScript to close the element, it stays visible after the animation is over. A solution would for example be to use a close button and for noscript users simply let it stay visible - this is also what happens with the noscript messages in Debusine).

So there are many “new” elements in HTML and a lot of “new” features of CSS. It makes sense to sometimes ask ourselves if instead of the solutions we know (or what a web search / some AI shows us as the most common solution) there might be some newer solution that was not there when the first choice was created. Using standardized solutions instead of custom libraries makes the software more maintainable. In web development I also prefer standardized elements over a third party library because they have usually better accessibility and UX.

In How Functional Programming Shaped (and Twisted) Frontend Development the author writes:

Consider the humble modal dialog. The web has <dialog>, a native element with built-in functionality: it manages focus trapping, handles Escape key dismissal, provides a backdrop, controls scroll-locking on the body, and integrates with the accessibility tree. It exists in the DOM but remains hidden until opened. No JavaScript mounting required.

[…]

you’ve trained developers to not even look for native solutions. The platform becomes invisible. When someone asks “how do I build a modal?”, the answer is “install a library” or “here’s my custom hook,” never “use <dialog>.”

Ahmad Alfy

20 October, 2025 05:28AM

October 19, 2025

hackergotchi for Colin Watson

Colin Watson

Mistaken dichotomies about dgit

In “Could the XZ backdoor have been detected with better Git and Debian packaging practices?”, Otto contrasts “git-buildpackage managed git repositories” with “dgit managed repositories”, saying that “the dgit managed repositories cannot incorporate the upstream git history and are thus less useful for auditing the full software supply-chain in git”.

Otto does qualify this earlier with “a package … that has not had the history recorded in dgit earlier”, but the last sentence of the section is a misleading oversimplification. It’s true for repositories that have been synthesized by dgit (which indeed was the focus of that section of Otto’s article), but it’s not true in general for repositories that are managed by dgit.

I suspect this was just slightly unclear writing, so I don’t want to nitpick here, but rather to take the opportunity to try to clear up some misconceptions around dgit that I’ve often heard at conferences and seen on mailing lists.

I’m not a dgit developer, although I’m a happy user of it and I’ve tried to help out in various design discussions over the years.

dgit and git-buildpackage sit at different layers

It seems very common for people to think of git-buildpackage and dgit as alternatives, as the example I quoted at the start of this article suggests. It’s really better to think of dgit as a separate and orthogonal layer.

You can use dgit together with tools such as git-buildpackage. In that case, git-buildpackage handles the general shape of your git history, such as helping you to import new upstream versions, and dgit handles gatewaying between the archive and git. The advantages become evident when you start using tag2upload, in which case you can just use git debpush to push a tag and the tag2upload service deals with building the source package and uploading it to the archive for you. This is true regardless of how you put your package’s git history together. (There’s currently a wrinkle around pristine-tar support, so at the moment I personally tend to use dgit push-source for new upstream versions and git debpush for new Debian revisions, since I haven’t yet convinced myself that I see no remaining value in pristine upstream tarballs.)

dgit supports complete history

If the maintainer has never used dgit, and so dgit clone synthesizes a repository based on the current contents of the Debian archive, then there’s indeed no useful history there; in that situation it doesn’t go back and import everything from the snapshot archive the way that gbp import-dscs --debsnap does.

However, if the maintainer uses dgit, then dgit’s view will include more history, and it’s absolutely possible for that to include complete upstream git history as well. Try this:

$ dgit clone man-db
canonical suite name for unstable is sid
fetching existing git history
last upload to archive: specified git info (debian)
downloading http://ftp.debian.org/debian//pool/main/m/man-db/man-db_2.13.1.orig.tar.xz...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 2060k  100 2060k    0     0  4643k      0 --:--:-- --:--:-- --:--:-- 4652k
downloading http://ftp.debian.org/debian//pool/main/m/man-db/man-db_2.13.1.orig.tar.xz.asc...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   833  100   833    0     0  16322      0 --:--:-- --:--:-- --:--:-- 16660
HEAD is now at 167835b0 releasing package man-db version 2.13.1-1
dgit ok: ready for work in man-db
$ git -C man-db log --graph --oneline | head
* 167835b0 releasing package man-db version 2.13.1-1
*   f7910493 New upstream release (2.13.1)
|\
| *   3073b72e Import man-db_2.13.1.orig.tar.xz
| |\
| | * 349ce503 Release man-db 2.13.1
| | * 0d6635c1 Update Russian manual page translation
| | * cbf87caf Update Italian translation
| | * fb5c5017 Update German manual page translation
| | * dae2057b Update Brazilian Portuguese manual page translation

That package uses git-dpm, since I prefer the way it represents patches. But it works fine with git-buildpackage too:

$ dgit clone isort
canonical suite name for unstable is sid
fetching existing git history
last upload to archive: specified git info (debian)
downloading http://ftp.debian.org/debian//pool/main/i/isort/isort_7.0.0.orig.tar.gz...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  786k  100  786k    0     0  1772k      0 --:--:-- --:--:-- --:--:-- 1774k
HEAD is now at f812aae releasing package isort version 7.0.0-1
dgit ok: ready for work in isort
$ git -C isort log --graph --oneline | head
* f812aae releasing package isort version 7.0.0-1
*   efde62f Update upstream source from tag 'upstream/7.0.0'
|\
| * 9694f3d New upstream version 7.0.0
* | 9cbfe0b releasing package isort version 6.1.0-1
* | 5423ffe Mark isort and python3-isort Multi-Arch: foreign
* | 5eaf5bf Update upstream source from tag 'upstream/6.1.0'
|\|
| * edafbfc New upstream version 6.1.0
* |   aedfd25 Merge branch 'debian/master' into fix992793

If you look closely you’ll see another difference here: the second only includes one commit representing the new upstream release, and doesn’t have complete upstream history. This doesn’t represent a difference between git-dpm and git-buildpackage. Both tools can operate in both ways: for example, git-dpm import-new-upstream --parent and gbp import-orig --upstream-vcs-tag do broadly similar things, and something like gbp import-dscs --debsnap --upstream-vcs-tag='%(version)s' can be used to do a bulk import provided that upstream’s tags are named consistently enough. This is not generally the default because adding complete upstream history requires extra setup: the maintainer has to add an extra git remote pointing to upstream and select the correct tag when importing a new version, and some upstreams forget to push git tags or don’t have the sort of consistency you might want.

The Debian Python team’s policy says that “Complete upstream Git history should be avoided in the upstream branch”, which is why the isort history above looks the way it does. I don’t love this because I think the results are less useful, but I understand why it’s there: in a moderately large team maintaining thousands of packages, getting everyone to have the right git remotes set up would be a recipe for frustrating inconsistency.

However, in packages I maintain myself, I strongly value having complete upstream history in order to make it easier to debug problems, and I think it makes things a bit more transparent to auditors too, so I’m willing to go to a little extra work to make that happen. Doing that is completely compatible with using dgit.

19 October, 2025 12:04PM by Colin Watson

October 18, 2025

Julian Andres Klode

Sound Removals

Problem statement

Currently if you have an automatically installed package A (= 1) where

  • A (= 1) Depends B (= 1)
  • A (= 2) Depends B (= 2)

and you upgrade B from 1 to 2; then you can:

  1. Remove A (= 1)
  2. Upgrade A to version 2

If A was installed by a chain initiated by Recommends (say X Rec Y, Y Depends A), the solver sometimes preferred removing A (and anything depending on it until it got).

I have a fix pending to introduce eager Recommends which fixes the practical case, but this is still not sound.

In fact we can show that the solver produces the wrong result for small minimal test cases, as well as the right result for some others without the fix (hooray?).

Ensuring sound removals is more complex, and first of all it begs the question: When is a removal sound? This, of course, is on us to define.

An easy case can be found in the Debian policy, 7.6.2 “Replacing whole packages, forcing their removal”:

If B (= 2) declares a Conflicts: A (= 1) and Replaces: A (= 1), then the removal is valid. However this is incomplete as well, consider it declares Conflicts: A (< 1) and Replaces: A (< 1); the solution to remove A rather than upgrade it would still be wrong.

This indicates that we should only allow removing A if the conflicts could not be solved by upgrading it.

The other case to explore is package removals. If B is removed, A should be removed as well; however it there is another package X that Provides: B (= 1) and it is marked for install, A should not be removed. That said, the solver is not allowed to install X to satisfy the depends B (= 1) - only to satisfy other dependencies [we do not want to get into endless loops where we switch between alternatives to keep reverse dependencies installed].

Proposed solution

To solve this, I propose the following definition:

Definition (sound removal): A removal of package P is sound if either:

  1. A version v is installed that package-conflicts with B.
  2. A package Q is removed and the installable versions of P package-depends on Q.

where the other definitions are:

Definition (installable version): A version v is installable if either it is installed, or it is newer than an installed version of the same package (you may wish to change this to accomodate downgrades, or require strict pinning, but here be dragons).

Definition (package-depends): A version v package-depends on a package B if either:

  1. there exists a dependency in v that can be solved by any version of B, or
  2. there exists a package C where v package-depends C and any (c in C) package-depends B (transitivity)

Definition (package-conflicts): A version v package-conflicts with an installed package B if either:

  1. it declares a conflicts against an installable version of B; or
  2. there exists a package C where v package-conflicts C, and b package-depends C for installable versions b.

Translating this into a (modified) SAT solver

One approach may be to implement the logic in the conflict analysis that drives backtracking, i.e. we assume a package A and when we reach not A, we analyse if the implication graph for not A constitutes a sound removal, and then replace the assumption A with the assumption A or "learned reason.

However, while this seems a plausible mechanism for a DPLL solver, for a modern CDCL solver, it’s not immediately evident how to analyse whether not A is sound if the reason for it is a learned clause, rather than a problem clause.

Instead we propose a static encoding of the rules into a slightly modified SAT solver:

Given c1, …, cn that transitive-conflicts A and D1, …, Dn that A package-depends on, introduce the rule:

A unless c1 or c2 or ... cn ... or not D1 or not D2 ... or not Dn

Rules of the form A... unless B... - where A... and B... are CNF - are intuitively the same as A... or B..., however the semantic here is different: We are not allowed to select B... to satisfy this clause.

This requires a SAT solver that tracks a reason for each literal being assigned, such as solver3, rather than a SAT solver like MiniSAT that only tracks reasons across propagation (solver3 may track A depends B or C as the reason for B without evaluating C, whereas MiniSAT would only track it as the reason given not C).

Is it actually sound?

The proposed definition of a sound removal may still proof unsound as I either missed something in the conclusion of the proposed definition that violates my goal I set out to achieve, or I missed some of the goals.

I challenge you to find cases that cause removals that look wrong :D

18 October, 2025 07:37PM

October 17, 2025

hackergotchi for David Bremner

David Bremner

Hibernate on the pocket reform 13/n

Context

Some progress upstream

Recently Sebastian Reichel at Collabora [1] has made a few related commits, apparently inspired in part by my kvetching on this blog.

Disconnecting and reconnecting PCI busses

At some point I noticed error message about the nvme device on resume. I then learned how to disconnect and reconnect PCI buses in Linux. I ended up with something like the following. At least the PCI management seems to work. I can manually disconnect all the PCI busses and rescan to connect them again on a running system. It presumably helps that I am not using the nvme device in this system.

set -x
echo platform >  /sys/power/pm_test
echo reboot > /sys/power/disk
rmmod mt76x2u
sleep 2
echo 1 | tee /sys/bus/pci/devices/0003:30:00.0/remove
sleep 2
echo 1 | tee /sys/bus/pci/devices/0004:41:00.0/remove
sleep 2
echo 1 | tee /sys/bus/pci/devices/0004:40:00.0/remove
sleep 2
echo LSPCI:
lspci -t
sleep 2
echo disk >  /sys/power/state
sleep 2
echo 1 | tee /sys/bus/pci/rescan
sleep 2
modprobe mt76x2u

Minimal changes to upstream

With the ongoing work at collabora I decided to try a minimal patch stack to get the pocket reform to boot. I added the following 3 commits (available from [3]).

09868a4f2eb (HEAD -> reform-patches) copy pocket-reform dts from reform-debian-packages
152e2ae8a193 pocket/panel: sleep fix v3
18f65da9681c add-multi-display-panel-driver

It does indeed boot and seems stable.

$ uname -a
Linux anthia 6.18.0-rc1+ #19 SMP Thu Oct 16 11:32:04 ADT 2025 aarch64 GNU/Linux

Running the hibernation script above I get no output from the lspci, but seemingly issues with PCI coming back from hibernate:

[  424.645109] PM: hibernation: Allocated 361823 pages for snapshot
[  424.647216] PM: hibernation: Allocated 1447292 kbytes in 3.23 seconds (448.07 MB/s)
[  424.649321] Freezing remaining freezable tasks
[  424.654767] Freezing remaining freezable tasks completed (elapsed 0.003 seconds)
[  424.661070] rk_gmac-dwmac fe1b0000.ethernet end0: Link is Down
[  424.740716] rockchip-dw-pcie a40c00000.pcie: Failed to receive PME_TO_Ack
[  424.742041] PM: hibernation: hibernation debug: Waiting for 5 second(s).
[  430.074757] pci 0004:40:00.0: [1d87:3588] type 01 class 0x060400 PCIe Root Port
F�F���&�Zn�[� watchdog: CPU4: Watchdog detected hard LOCKUP on cpu 5
[  456.039004] Modules linked in: xt_CHECKSUM xt_tcpudp nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat x_tables bridge stp llc nf_tables aes_neon_bs aes_neon_blk ccm dwmac_rk binfmt_misc mt76x2_common mt76x02_usb mt76_usb mt76x02_lib mt76 mac80211 rk805_pwrkey snd_soc_tlv320aic31xx snd_soc_simple_card reform2_lpc(OE) libarc4 rockchip_saradc industrialio_triggered_buffer kfifo_buf industrialio cfg80211 rockchip_thermal rockchip_rng hantro_vpu cdc_acm v4l2_vp9 v4l2_jpeg rockchip_rga rfkill snd_soc_rockchip_i2s_tdm videobuf2_dma_sg v4l2_h264 panthor snd_soc_audio_graph_card drm_gpuvm snd_soc_simple_card_utils drm_exec evdev joydev dm_mod nvme_fabrics efi_pstore configfs nfnetlink autofs4 ext4 crc16 mbcache jbd2 btrfs blake2b_generic xor xor_neon raid6_pq mali_dp snd_soc_meson_axg_toddr snd_soc_meson_axg_fifo snd_soc_meson_codec_glue panfrost drm_shmem_helper gpu_sched ao_cec_g12a meson_vdec(C) videobuf2_dma_contig videobuf2_memops v4l2_mem2mem videobuf2_v4l2 videodev
[  456.039060]  videobuf2_common mc dw_hdmi_i2s_audio meson_drm meson_canvas meson_dw_mipi_dsi meson_dw_hdmi mxsfb mux_mmio panel_edp imx_dcss ti_sn65dsi86 nwl_dsi mux_core pwm_imx27 hid_generic usbhid hid xhci_plat_hcd onboard_usb_dev xhci_hcd nvme nvme_core snd_soc_hdmi_codec snd_soc_core nvme_keyring nvme_auth hkdf snd_pcm_dmaengine snd_pcm snd_timer snd soundcore fan53555 rtc_pcf8523 micrel phy_package stmmac_platform stmmac pcs_xpcs rk808_regulator phylink sdhci_of_dwcmshc mdio_devres dw_mmc_rockchip of_mdio sdhci_pltfm phy_rockchip_usbdp fixed_phy dw_mmc_pltfm fwnode_mdio typec phy_rockchip_naneng_combphy phy_rockchip_samsung_hdptx pwm_rockchip sdhci dwc3 libphy dw_wdt dw_mmc ehci_platform rockchip_dfi mdio_bus cqhci ulpi ohci_platform ehci_hcd udc_core ohci_hcd rockchipdrm phy_rockchip_inno_usb2 usbcore dw_hdmi_qp analogix_dp dw_mipi_dsi cpufreq_dt dw_mipi_dsi2 i2c_rk3x usb_common drm_dp_aux_bus [last unloaded: mt76x2u]
[  456.039111] Sending NMI from CPU 4 to CPUs 5:
[  471.942262] page_pool_release_retry() stalled pool shutdown: id 9, 2 inflight 60 sec
[  532.989611] page_pool_release_retry() stalled pool shutdown: id 9, 2 inflight 121 sec

This does look like some progress, probably thanks to Sebastien. Comparing with the logs in hibernate-pocket-12, the resume process is not bailing out complaining about PHY.

Attempt to reapply PCI reset patches

Following the procedure in hibernate-pocket-12, I attempted to re-apply the pci reset patches [2]. In particular I followed the hints output by b4.

Unfortunately there are too many conflicts now for me to sensibly resolve.


  1. https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux.git#rockchip-devel

  2. https://lore.kernel.org/all/20250715-pci-port-reset-v6-0-6f9cce94e7bb@oss.qualcomm.com/#r

  3. https://salsa.debian.org/bremner/collabora-rockchip-3588#reform-patches

17 October, 2025 10:50PM

Hibernate on the pocket reform 12/n

Context

Update to latest rockchip-devel

For some reason I decided to try re-applying the PCI series. Good news: the pci series finally applies cleanly.

$ git fetch collabora && git switch -c tmp collabora  # [1]
$ b4 am 20250715-pci-port-reset-v6-0-6f9cce94e7bb@oss.qualcomm.com
$ git switch reform-patches  # [2]
$ git rebase -i tmp
  1. https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux.git#rockchip-devel
  2. https://salsa.debian.org/bremner/collabora-rockchip-3588#reform-patches

Rebuild the kernel

$ cp /boot/config-6.17.0-rc7+ .config
$ make olddefconfig
$ yes '' | make localmodconfig
$ make KBUILD_IMAGE=arch/arm64/boot/Image bindeb-pkg -j$(nproc)

try the hibernation test, again

Running the following test script

set -x
echo platform >  /sys/power/pm_test
echo reboot > /sys/power/disk
sleep 2
rmmod mt76x2u
sleep 2
echo disk >  /sys/power/state
sleep 2
modprobe mt76x2u

Initially there is some output like this

[  151.752683] rockchip-dw-pcie a40c00000.pcie: Failed to receive PME_TO_Ack
[  151.754035] PM: hibernation: hibernation debug: Waiting for 5 second(s).
[  157.821584] rockchip-dw-pcie a40c00000.pcie: Phy link never came up
[  157.822139] rockchip-dw-pcie a40c00000.pcie: fail to resume
[  157.822636] rockchip-dw-pcie a40c00000.pcie: PM: dpm_run_callback(): genpd_restore_noirq returns -110
[  157.823442] rockchip-dw-pcie a40c00000.pcie: PM: failed to restore noirq: error -110

A small amount of detective work suggests that a40c00000.pcie corresponds to the first PCI bridge on the rk3588 SOC.

$ ls -l /sys/bus/pci/devices
total 0
lrwxrwxrwx 1 root root 0 Sep 23 10:32 0003:30:00.0 -> ../../../devices/platform/a40c00000.pcie/pci0003:30/0003:30:00.0
lrwxrwxrwx 1 root root 0 Sep 23 10:32 0004:40:00.0 -> ../../../devices/platform/a41000000.pcie/pci0004:40/0004:40:00.0
lrwxrwxrwx 1 root root 0 Sep 23 10:32 0004:41:00.0 -> ../../../devices/platform/a41000000.pcie/pci0004:40/0004:40:00.0/0004:41:00.0

Then after a pause,

[ 1032.039237] watchdog: CPU5: Watchdog detected hard LOCKUP on cpu 6
[ 1032.039778] Modules linked in: xt_CHECKSUM xt_tcpudp nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat x_tables bridge stp llc nf_tables aes_neon_bs aes_neon_blk ccm dwmac_rk binfmt_misc mt76x2_common mt76x02_usb mt76_usb mt76x02_lib mt76 rk805_pwrkey snd_soc_tlv320aic31xx snd_soc_simple_card mac80211 rockchip_saradc reform2_lpc(OE) industrialio_triggered_buffer libarc4 kfifo_buf cfg80211 industrialio rockchip_thermal rockchip_rng cdc_acm rfkill snd_soc_rockchip_i2s_tdm hantro_vpu rockchip_rga panthor v4l2_vp9 v4l2_jpeg snd_soc_audio_graph_card videobuf2_dma_sg v4l2_h264 drm_gpuvm snd_soc_simple_card_utils drm_exec evdev joydev dm_mod nvme_fabrics efi_pstore configfs nfnetlink autofs4 ext4 crc16 mbcache jbd2 btrfs blake2b_generic xor xor_neon raid6_pq mali_dp snd_soc_meson_axg_toddr snd_soc_meson_axg_fifo snd_soc_meson_codec_glue panfrost drm_shmem_helper gpu_sched ao_cec_g12a meson_vdec(C) videobuf2_dma_contig videobuf2_memops v4l2_mem2mem videobuf2_v4l2 videodev
[ 1032.039834]  videobuf2_common mc dw_hdmi_i2s_audio meson_drm meson_canvas meson_dw_mipi_dsi meson_dw_hdmi mxsfb mux_mmio panel_edp imx_dcss ti_sn65dsi86 nwl_dsi mux_core pwm_imx27 hid_generic usbhid hid onboard_usb_dev nvme nvme_core nvme_keyring nvme_auth snd_soc_hdmi_codec snd_soc_core xhci_plat_hcd xhci_hcd snd_pcm_dmaengine snd_pcm snd_timer snd soundcore rtc_pcf8523 fan53555 micrel phy_package stmmac_platform stmmac pcs_xpcs phylink mdio_devres rk808_regulator of_mdio sdhci_of_dwcmshc fixed_phy sdhci_pltfm fwnode_mdio libphy sdhci phy_rockchip_usbdp dw_mmc_rockchip dw_mmc_pltfm typec phy_rockchip_naneng_combphy pwm_rockchip dw_wdt phy_rockchip_samsung_hdptx dwc3 cqhci dw_mmc mdio_bus rockchip_dfi ehci_platform rockchipdrm ulpi ehci_hcd dw_hdmi_qp ohci_platform udc_core ohci_hcd analogix_dp dw_mipi_dsi i2c_rk3x cpufreq_dt usbcore phy_rockchip_inno_usb2 dw_mipi_dsi2 drm_dp_aux_bus usb_common [last unloaded: mt76x2u]
[ 1032.039886] Sending NMI from CPU 5 to CPUs 6:

previous episode

17 October, 2025 10:36PM

hackergotchi for Sean Whitton

Sean Whitton

Southern Biscuits with British ingredients

I miss the US more and more, and have recently been trying to perfect Southern Biscuits using British ingredients. It took me eight or nine tries before I was consistently getting good results. Here is my recipe.

Ingredients

  • 190g plain flour
  • 60g strong white bread flour
  • 4 tsp baking powder
  • ¼ tsp bicarbonate of soda
  • 1 tsp cream of tartar (optional)
  • 1 tsp salt
  • 100g unsalted butter
  • 180ml buttermilk, chilled
    • If your buttermilk is thicker than the consistency of ordinary milk, you’ll need around 200ml.
  • extra buttermilk for brushing

Method

  1. Slice and then chill the butter in the freezer for at least fifteen minutes.
  2. Preheat oven to 220°C with the fan turned off.
  3. Twice sieve together the flours, leaveners and salt. Some salt may not go through the sieve; just tip it back into the bowl.
  4. Cut cold butter slices into the flour with a pastry blender until the mixture resembles coarse crumbs: some small lumps of fat remaining is desirable. In particular, the fine crumbs you are looking for when making British scones are not wanted here. Rubbing in with fingertips just won’t do; biscuits demand keeping things cold even more than shortcrust pastry does.
  5. Make a well in the centre, pour in the buttermilk, and stir with a metal spoon until the dough comes together and pulls away from the sides of the bowl. Avoid overmixing, but I’ve found that so long as the ingredients are cold, you don’t have to be too gentle at this stage and can make sure all the crumbs are mixed in.
  6. Flour your hands, turn dough onto a floured work surface, and pat together into a rectangle. Some suggest dusting the top of the dough with flour, too, here.
  7. Fold the dough in half, then gather any crumbs and pat it back into the same shape. Turn ninety degrees and do the same again, until you have completed a total of eight folds, two in each cardinal direction. The dough should now be a little springy.
  8. Roll to about ½ inch thick.
  9. Cut out biscuits. If using a round cutter, do not twist it, as that seals the edges of the biscuits and so spoils the layering.
  10. Transfer to a baking sheet, placed close together (helps them rise). Flour your thumb and use it to press an indent into the top of each biscuit (helps them rise straight), brush with buttermilk.
  11. Bake until flaky and golden brown: about fifteen minutes.

Gravy

It turns out that the “pepper gravy” that one commonly has with biscuits is just a white/béchamel sauce made with lots of black pepper. I haven’t got a recipe I really like for this yet. Better is a “sausage gravy”; again this has a white sauce as its base, I believe. I have a vegetarian recipe for this to try at some point.

Variations

  • These biscuits do come out fluffy but not so flaky. For that you can try using lard instead of butter, if you’re not vegetarian (vegetable shortening is hard to find here).
  • If you don’t have a pastry blender and don’t want to buy one you can try not slicing the butter and instead coarsely grating it into the flour out of the freezer.
  • An alternative to folding is cutting and piling the layers.
  • You can try rolling out to 1–1½ inches thick.
  • Instead of cutting out biscuits you can just slice the whole piece of dough into equal pieces. An advantage of this is that you don’t have to re-roll, which latter also spoils the layering.
  • Instead of brushing with buttermilk, you can take them out after they’ve started to rise but before they’ve browned, brush them with melted butter and put them back in.

Notes

  • I’ve had more success with Dale Farm’s buttermilk than Sainsbury’s own. The former is much runnier.
  • Southern culture calls for biscuits to be made the size of cat’s heads.
  • Bleached flour is apparently usual in the South, but is illegal(!) here. This shouldn’t affect texture or taste but may make them look different.
  • American all-purpose flour has more gluten than our plain flour, hence the mix of plain and strong white, in a ratio of 3:1.
  • Baking powder in the US is usually double-acting but ours is always single-acting, so we need double quantities of that.

17 October, 2025 08:02PM

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

ML quacks: Combining duckdb and mlpack

A side project I have been working on a little since last winter and which explores extending duckdb with mlpack is now public at the duckdb-mlpack repo.

duckdb is an excellent ‘small’ (as in ‘runs as a self-contained binary’) database engine with both a focus on analytical payloads (OLAP rather than OLTP) and an impressive number of already bolted-on extensions (for example for cloud data access) delivered as a single-build C++ executable (or of course as a library used from other front-ends). mlpack is an excellent C++ library containing many/most machine learning algorithms, also built in a self-contained manner (or library) making it possible to build compact yet powerful binaries, or to embed (as opposed to other ML framework accessed from powerful but not lightweight run-times such as Python or R). The compact build aspect as well as the common build tools (C++, cmake) make these two a natural candidate for combining them. Moreover, duckdb is a champion of data access, management and control—and the complementary machine learning insights and predictions offered by mlpack are fully complementary and hence fit this rather well.

duckdb also has a very robust and active extension system. To use it, one starts from a template repository and its ‘use this template’ button, runs a script and can then start experimenting. I have now grouped my initial start and test functions into a separate repository duckdb-example-extension to keep the duckdb-mlpack one focused on the ‘extend to mlpack’ aspect.

duckdb-mlpack is right an “MVP”, i.e. a minimally viable product (or demo). It just runs the adaboost classifier but does so on any dataset fitting the ‘rectangular’ setup with columns of features (real valued) and a final column (integer valued) of labels. I had hope to use two select queries for both features and then labels but it turns a ‘table’ function (returning a table of data from a query) can only run one select *. So the basic demo, also on the repo README is now to run the following script (where the SELECT * FROM mlpack_adaboost((SELECT * FROM D)); is the key invocation of the added functionality):

#!/bin/bash

cat <<EOF | build/release/duckdb
SET autoinstall_known_extensions=1;
SET autoload_known_extensions=1; # for httpfs

CREATE TEMP TABLE Xd AS SELECT * FROM read_csv("https://mlpack.org/datasets/iris.csv");
CREATE TEMP TABLE X AS SELECT row_number() OVER () AS id, * FROM Xd;
CREATE TEMP TABLE Yd AS SELECT * FROM read_csv("https://mlpack.org/datasets/iris_labels.csv");
CREATE TEMP TABLE Y AS SELECT row_number() OVER () AS id, CAST(column0 AS double) as label FROM Yd;
CREATE TEMP TABLE D AS SELECT * FROM X INNER JOIN Y ON X.id = Y.id;
ALTER TABLE D DROP id;
ALTER TABLE D DROP id_1;
CREATE TEMP TABLE A AS SELECT * FROM mlpack_adaboost((SELECT * FROM D));

SELECT COUNT(*) as n, predicted FROM A GROUP BY predicted;
EOF

to produce the following tabulation / group by:

./sampleCallRemote.sh
Misclassified: 1
┌───────┬───────────┐
   n   │ predicted │
 int64 │   int32   │
├───────┼───────────┤
    50 │         0 │
    49 │         1 │
    51 │         2 │
└───────┴───────────┘
$

(Note that this requires the httpfs extension. So when you build from a freshly created extension repository you may be ‘ahead’ of the most recent release of duckdb by a few commits. It is easy to check out the most recent release tag (or maybe the one you are running for your local duckdb binary) to take advantage of the extensions you likely already have for that version. So here, and in the middle of October 2025, I picked v1.4.1 as I run duckdb version 1.4.1 on my box.)

There are many other neat duckdb extensions. The ‘core’ ones are regrouped here while a list of community extensions is here and here.

For this (still more minimal) extension, I added a few TODO items to the README.md:

  • More examples of model fitting and prediction
  • Maybe set up model serialization into table to predict on new data
  • Ideally: Work out how to SELECT from multiple tabels, or else maybe SELECT into temp. tables and pass temp. table names into routine
  • Maybe add mlpack as a git submodule

Please reach out if you are interested in working on any of this.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

17 October, 2025 06:07PM

October 16, 2025

Swiss JuristGate

Mathieu Parreaux claims FINMA knew since day one

In an extraordinary final LinkedIn post about the closure of his illegal legal insurance venture, the founder, Mathieu Parreaux claimed the Swiss financial regulator, FINMA, was aware of his business since the very beginning of the partnership structure in 2018. In other words, he alleges FINMA knew about it for five years.

To clarify, Mathieu Parreaux had originally created the business using the name Parreaux & Associés in October 2016. This name appears on LinkedIn but it was not in the business register. In 2018, he recruited the business partner Francois Thiébaud and formed the new entity Parreaux, Thiébaud SNC approximately eight weeks after Thiébaud had closed his previous business, a massage parlour in Le Locle.

LinkedIn alleges that Parreaux's final post has been edited a number of times. I attach a screenshot taken on 7 October 2025.

In paragraph five, Parreaux has written "After five years of discussions", implying that FINMA has been aware of his business and discussing it with him for five years.

My understanding of financial services is that anybody who wants to operate an investment or insurance scheme needs to obtain their licenses and permits before accepting the first payment from a customer. But I am not a jurist like Mathieu Parreaux, so what would I know?

Parreaux confirms that 20,000 clients were impacted. He claims that the government doesn't care about the clients, which included my own business the Software Freedom Institute.

He claims to care about the interests of clients and offers to "support them until the end, at our own expense". He does not share any financial information about assets and running costs of the business so we can not determine whether this proposition was credible or not.

Then again, FINMA only published their side of the story more than six months later, they redacted over ninety percent of paragraphs and they even redacted the dates and company names from the judgment. Therefore, the information that was eventually disclosed from FINMA is not sufficient for anybody to know who is right and who is wrong.

Here is a translation of Parreaux's final LinkedIn post in English:

The inside story of a true scandal... Here is the story of a Swiss man currently shocked by a country he loves with all his heart.

For five years, through Justicia SA, we have been offering a legal product that democratizes access to the law. The concept is simple: you pay a legal subscription of a few hundred francs on average, and our firm's services are available to you without any additional charges or service exclusions.

A mini-revolution for the legal field and the billing method.

A huge revolution for the legal insurance sector, which we have tackled head-on.

After five years of discussions, FINMA sent us a letter containing a decision to immediately close our company because it deemed us to be an insurance company and that we did not comply with certain conditions related to this activity.

While this is perfectly conceivable, FINMA gives us no deadline to comply with these conditions. Incredibly, FINMA also removes any suspensive effect on any legal action.

This means they're deciding that their decision is immediate, and that even if we win an appeal in two years, everything will have been reduced to dust, leaving us with nothing but a possible "Sorry, guys! You can start all over again."

It gets even worse. The approximately 20,000 clients we currently represent are left without assistance. With a complete shutdown of our business, our government doesn't care about these thousands of clients who are currently in need.

A profoundly humane choice would have been, at the very least, to order a closure in a year, at the end of the clients' contractual period, so that we could support them until the end, at our own expense.

We are dealing with a total disregard not only for innovative companies, but also for individuals who need our assistance.

When we weigh these facts against our state's tolerance for companies that are rotten to the core, one thing stands out: in our democracy, the rotten giants will always fare better than the innovative little ones.

Despite these events, this allows us to fill the gaps that existed. With our motivation, our determination, and our influential partners, we will return.

We will return. More numerous. More motivated. Stronger.

Mathieu Elias Parreaux, LinkedIn, FINMA

16 October, 2025 11:30AM

October 14, 2025

hackergotchi for Gunnar Wolf

Gunnar Wolf

Can a server be just too stable?

One of my servers at work leads a very light life: it is our main backups server (so it has a I/O spike at night, with little CPU involvement) and has some minor services running (i.e. a couple of Tor relays and my personal email server — yes, I have the authorization for it 😉). It is a very stable machine… But today I was surprised:

As I am about to migrate it to Debian 13 (Trixie), naturally, I am set to reboot it. But before doing so:

$ w
 12:21:54 up 1048 days, 0 min,  1 user,  load average: 0.22, 0.17, 0.17
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU  WHAT
gwolf             192.168.10.3     12:21           0.00s  0.02s sshd-session: gwolf [priv]

Wow. Did I really last reboot this server on December 1 2022?

(Yes, I know this might speak bad of my security practices, as there are several kernel updates I never applied, even having installed the relevant packages. Still, it got me impressed 😉)

Debian. Rock solid.

Debian Rocks

14 October, 2025 06:22PM

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

qlcal 0.0.17 on CRAN: Regular Update

The seventeenth release of the qlcal package arrivied at CRAN today, once again following a QuantLib release as 1.40 came out this morning.

qlcal delivers the calendaring parts of QuantLib. It is provided (for the R package) as a set of included files, so the package is self-contained and does not depend on an external QuantLib library (which can be demanding to build). qlcal covers over sixty country / market calendars and can compute holiday lists, its complement (i.e. business day lists) and much more. Examples are in the README at the repository, the package page, and course at the CRAN package page.

This releases mainly synchronizes qlcal with the QuantLib release 1.40. Only one country calendar got updated; the diffstat looks larger as the URL part of the copyright got updated throughout. We also updated the URL for the GPL-2 badge: when CRAN checks this, they always hit a timeout as the FSF server possibly keeps track of incoming requests; we now link to version from the R Licenses page to avoid this.

Changes in version 0.0.17 (2025-07-14)

  • Synchronized with QuantLib 1.40 released today

  • Calendar updates for Singapore

  • URL update in README.md

Courtesy of my CRANberries, there is a diffstat report for this release. See the project page and package documentation for more details, and more examples.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

14 October, 2025 05:55PM

October 13, 2025

hackergotchi for Jonathan McDowell

Jonathan McDowell

onak 0.6.4 released

A bit delayed in terms of an announcement, but last month I tagged a new version of onak, my OpenPGP compatible keyserver. It’s been 2 years since the last release, and this is largely a bunch of minor fixes to make compilation under Debian trixie with more recent CMake + GCC versions happy.

OpenPGP v6 support, RFC9580, hasn’t made it. I’ve got a branch which adds it, but a lack of keys to do any proper testing with, and no X448 support implemented, mean I’m not ready to include it in a release yet. The plan is that’ll land for 0.7.0 (along with some backend work), but no idea when that might be.

Available locally or via GitHub.

0.6.4 - 7th September 2025

  • Fix building with CMake 4.0
  • Fixes for building with GCC 15
  • Rename keyd(ctl) to onak-keyd(ctl)

13 October, 2025 06:31PM

hackergotchi for Wouter Verhelst

Wouter Verhelst

RPM and ECDSA GPG keys

Dear lazyweb,

At work, we are trying to rotate the GPG signing keys for the Linux packages of the eID middleware

We created new keys, and they will be installed on all Linux machines that have the eid-archive package installed soon (they were already supposed to be, but we made a mistake).

Running some tests, however, I have a bit of a problem:

[wouter@rhel rpm-gpg]$ sudo rpm --import RPM-GPG-KEY-BEID-RELEASE
[wouter@rhel rpm-gpg]$ sudo rpm --import RPM-GPG-KEY-BEID-RELEASE-2025
fout: RPM-GPG-KEY-BEID-RELEASE-2025: key 1 import failed.
[wouter@rhel rpm-gpg]$ sudo rpm --import RPM-GPG-KEY-BEID-CONTINUOUS

This is on RHEL9.

The only difference between the old keys and the new one, apart of course from the fact that the old one is, well, old, is that the old one uses the RSA algorithm whereas the new one uses ECDSA on the NIST P-384 curve (the same algorithm as the one used by the eID card).

Does RPM not support ECDSA keys? Does anyone know where this is documented?

(Yes, I probably should have tested this before publishing the new key, but this is where we are)

13 October, 2025 09:51AM

Russell Coker

WordPress Spam Users

Just over a year ago I configured my blog to only allow signed in users to comment to reduce spam [1]. This has stopped all spam comments, it was even more successful than expected but spammers keep registering accounts. I’ve now got almost 5000 spam accounts, an average of more than 10 per day. I don’t know why they keep creating them without trying to enter comments. At first I thought that they were trying to assemble a lot of accounts for a deluge of comment spam but that hasn’t happened.

There are some WordPress plugins for bulk deletion of users but I couldn’t find one with support for “delete all users who haven’t submitted a comment”. So I do it a page at a time, but of course I don’t want to do it 100 at a time so I used the below SQL to change it to 400 at a time. I initially tried larger numbers like 2000 but got Chrome timeouts when trying to click the check-box to select all users. From experimenting it seems that the time taken to check that is worse than linear. Doing it for 2000 users is obviously much more than 5* the duration of doing it for 400. 800 users was one attempt which resulted in it being possible to select them all but then it gave an error about the URL being too long when it came to actually delete them. After a binary search I found that 450 was too many but 400 worked. So now it’s 12 operations to delete all the spam accounts. Each bulk delete operation is 5 GUI operations so it’s 60 operations to delete 15 months of spam users. This is annoying, but less than the other problems of spam.

UPDATE `wp_usermeta` SET `meta_value` = 400 WHERE `user_id` = 2 AND `meta_key` = 'users_per_page';

Deleting the spam users reduced the size of the backup (zstd -9 of a mysql dump) for my blog by 6.5%. Then changing from zstd -9 to -19 reduced it by another 13%. After realising this difference I configured all my mysql backups to be compressed with zstd -19, this will make a difference on the system with over 30G of zstd compressed mysql backups.

13 October, 2025 04:14AM by etbe

October 12, 2025

Swiss JuristGate

Mathieu Elias Parreaux declared bankrupt in Switzerland

In December 2023, the bankruptcy procedure was started against Mathieu Elias Parreaux.

This is a procedure for the bankruptcy of Mr Parreaux in a personal capacity. His business had already been liquidated prior to this.

The Swiss gazette published a provisional alert.

In April 2024, the gazette contained a call to creditors.

In June 2024, the gazette announced they had completed an inventory of assets and a plan for dividing the assets between creditors.

There have been ongoing reports that Mathieu Parreaux is giving people advice about insurance and investments, especially in the crypto market. Anybody obtaining financial advice from Parreaux would be wise to ask him to explain the full circumstances of his own bankruptcy.

In particular, we need to understand whether Parreaux's businesses failed due to his own mistakes or due to wrongdoing by a third party such as an employee or a key supplier to his business.

Normally when a business fails, the personal assets of the directors and shareholders are not subject to the business bankruptcy. We need to understand what is different in this case, why was Mr Parreaux placed in bankruptcy in a personal capacity? For example, was he subject to judgments concerning criminal conduct or a civil judgment to compensate third parties?

Mr Parreaux needs to publish all the documents relating to his bankrupcty, including all the judgments and decisions from FINMA and the debt collection office.

Mr Parreaux's businesses have included a number of people who appear to be close colleagues and family members. The possible family members with the name Parreaux are Arnaud Parreaux, Marla Sofia Parreaux, Didier Parreaux and Hodan Parreaux-Hussein.

Naturally, creditors would like to know if any of the family members or friends have received salary payments or other benefits that could represent a transfer of assets out of the businesses prior to their liquidation.

12 October, 2025 05:00PM

Swiss pimp usurping reputation of legendary Tissot boss Francois Thiébaud from France (BaselWorld, SWATCH Group SA)

Francois Thiébaud was born some time between 1946 and 1947 in France-Compte, a region of the French Jura that borders Switzerland. He completed a law degree at the University of Besancon and an MBA at l'Institut de contrôle de gestion de Paris (a reknowned French business school).

In 1970, Francois Thiébaud started his career in watchmaking in Morteau, France. Morteau is a scenic village in the Jura mountains approximately thirty kilometers from the Swiss border. Many French people who live in these border towns travel to work in Switzerland every day. In French, these cross-border workers are known as Frontalièrs.

The quartz crisis rocked the watchmaking industry in the 1970s. In 1979, Monsieur Thiébaud went to work for Swiss company Breitling. He was responsible for positioning the Breitling brand as a watch for aviators, a marketing theme that remains strong today.

In 1996, Francois Thiébaud was recruited to lead the company Tissot, based in Le Locle, Swiss Jura. In 2000 he became president of the Tissot brand and in 2006 he became a member of the SWATCH Group Executive Management Board.

Under his tenure, Tissot output increased from 800,000 watches per year to over 4 million watches per year.

In 2002, the French-born Thiébaud was appointed president of the organizing committee for BaselWorld, the world-reknowned Swiss watch industry expo.

In 2015, Tissot achieved a major coup when they became timekeeper for America's National Basketball Association, the NBA.

Francois Thiébaud went to America for the contract-signing.

Francois Thiébaud, Tissot, SWATCH, NBA

 

Francois Thiébaud, Tissot, SWATCH, NBA

 

Thiébaud was interviewed by many media organizations, including Forbes. The journalist prefixes his responses with the initials FT.

Francois Thiébaud

 

Swiss business records tell us that in March 2017, in Le Locle, the village everybody knows as the home of Tissot, Francois Thiébaud obtained a business license for a bar and massage parlour trading as El Paraiso at 42 rue Girardet.

Francois Thiébaud, El Paraiso, Justicia, SAFE

 

To find the record, visit the web site of the Swiss government gazette. For some reason, there are two search boxes on the page. One searches all the records before 2 September 2018 while the other box searches for all records after that date. Type the name of Mr Thiébaud and the name of the place, Francois Thiébaud Locle. Here is a screenshot of the search results:

Francois Thiébaud, Le Locle, El Paraiso, Massage, Brothel, Prostitution

 

Francois Thiébaud, Le Locle, El Paraiso, Massage, Brothel, Prostitution, Tissot, Certina, Mido

 

Clicking the name of El Paraiso returns the business registration certificate from 4 April 2017.

Francois Thiébaud, Le Locle, El Paraiso, Massage, Brothel, Prostitution

 

On 10 May 2017 the news website ArcInfo published a report about Francois Thiébaud complaining that the local authority was selling the building out underneath him. The report emphasizes the use of the building for prostitution and they tell us that Mr Thiébaud is from Vevey which is more than two hours away from Le Locle.

In December 2017 Thiébaud canceled the business license for El Paraiso.

In 2018, various news reports appeared telling us that Francois Thiebaud of Vevey had merged his "services business" with the legal practice Parreaux & Associés to form the new business Parreaux, Thiébaud & Partners.

The news report about prostitution has a note at the bottom in bold. They mention that on 11 June 2018 they decided to redact the name in the report and replace it with the initials F.T., like the news report from Forbes. They are not talking about the same F.T.

If the journalists at ArcInfo realized Francois Thiébaud from Vevey was usurping the reputation of the French Francois Thiébaud on the board of SWATCH Group, why did the Swiss financial regulator FINMA fail to notice the same phenomena?

When Francois Thiébaud from Vevey pulled out of the massage business to become the partner in a legal fees insurance firm, who was responsible for the probity checks?

At the time, Mathieu Parreaux, the other partner in the legal fees insurance business, was still working as the Greffier, with judge-like powers, in a tribunal in Canton Valais. The snapshot of his biography tells us he had previously worked as a KYC (Know-Your-Client) officer at Safra Sarasin and Audi Bank in Geneva. Given this experience, it looks likely he would know his new business partner, Francois Thiébaud from Vevey had previously run a massage parlour usurping the reputation of another prominent businessman. If Parreaux knew all of that and if Francois Thiébaud from Vevey had no prior experience in the legal or insurance industries then we can contemplate the possibility that Mathieu Parreaux only chose this other man as a business partner to continue the tradition of usurping the reputation of the real Francois Thiébaud from France, the world-reknowned president of Tissot.

Even when Parreaux seemed to know FINMA was about to shut him down in 2023, a new business entity, Justiva SA was created with Francois Thiébaud from Vevey as the company director. They seemed to know that the name Thiébaud was good as gold.

Whenever you open a new bank account, purchase an insurance policy or take out a loan, the firm asks you to provide all kinds of sensitive documents about your identity and financial history. Behind the scenes, KYC officers like Mathieu Parreaux have access to all those documents and huge databases about all of us. Wouldn't it be creepy if these men were using that data to usurp our identities for their own purposes?

I can't help remembering the ongoing saga of Matthias Kirschner using the name FSFE to usurp the reputation of the real Dr Richard Stallman and real FSF in Boston.

By using Francois Thiébaud from Vevey to usurp the reputation of Francois Thiébaud from France, the world-reknowned president of Tissot, the Swiss jurists are doing something that smells like identity theft.

In fact, because Francois Thiébaud, president of Tissot is really from France, he will always be a Frontalier. He may be one of the most successful Frontaliers in Switzerland. The Swiss jurists have used the reputation of the most successful Frontalier in Switzerland to deceive other Frontaliers to purchase a worthless insurance. Moreover, they even deceived a French woman to quit her job in a real insurance company to come and work for the illegal legal fees insurance in Geneva.

They tricked all those Frontaliers by using the name of Francois Thiébaud, a French man, the most successful Frontalier.

In 2019, the Swatch group chose not to participate in BaselWorld, the world's largest watch fair, despite the fact Thiébaud had been president of the BaselWorld committee for 20 years. Was he afraid to go out in public after the other Francois Thiébaud from Vevey had started sullying his reputation?

Francois Thiébaud of Tissot and BaselWorld is not the same Francois Thiébaud who decided to run a massage parlour in Le Locle. It is a long drive from Vevey to Le Locle. If you were starting your first business, why would you do so in a location so far from your home?

Another news report appears about the prostitution in 2021. The title tells us that Le Locle is no longer the capital of love for hire (Prostitution: Le Locle n’est plus la capitale des amours tarifées).

Francois Thiébaud is a French citizen. He grew up very close to the border with Switzerland and dedicated his entire life and career to the watch industry.

Francois Thiébaud of Vevey is a Swiss citizen. He is a totally different person. It is important to think about why he chose to operate that type of business and why he chose to do so in Le Locle. Was he hoping that anybody searching for his name in Google would be confused by all the news reports about Francois Thiébaud at Tissot? In other words, a Swiss Thiébaud obtaining obscurity for himself by operating in the shadow of the French Thiébaud, a French man who appears to be superior to the Swiss man in every way?

Can somebody use their own name for identity theft? It is a complicated subject.

The Swiss Thiébaud decided to move from prostitution into the legal insurance services industry. These are industries that pretend to practice the highest standards of integrity.

Comparing the business experience and qualifications of Mathieu Parreaux and Francois Thiébaud, we can see that Parreaux had graduated with a law degree and he had worked in a number of legal and banking jobs. At the time when Parreaux and Thiébaud decided to join forces, Parreaux was the greffier for a tribunal in Monthey.

Did Parreaux enter into the partnership with the more junior Thiébaud for any reason other than to usurp the name and reputation of the French Thiébaud on the board of SWATCH Group?

More significantly, in early 2023, as FINMA progressed unacceptably slowly in their investigation of Parreaux, Thiébaud & Partners, we can see the partners created a new business entity using only Mr Thiébaud. When FINMA shut down Parreaux, Thiébaud & Partners, Justicia SA in April 2023, the shelf company under Mr Thiébaud was renamed to Justiva SA and began trying to recruit the former clients of the shuttered enterprise.

Why did FINMA only issue a ban against one partner, Monsieur Parreaux and not both of them? Was FINMA afraid to create a ban order with the name Francois Thiébaud in it because it is the same name as Monsieur Thiébaud at Tissot? Or did FINMA decide not to ban the Swiss Thiébaud because they knew he wasn't really in charge, he was only there for his name?

The new entity Justiva SA continued operating for another eight or nine months. How could FINMA completely fail to notice it was there? Was FINMA fooled by the name of Francois Thiébaud as a director?

Parreaux, Thiébaud & Partners deceived Frontalièrs with promises of legal protection that they failed to fulfil. In the one case Arnaud Parreaux took to the federal court, he argued against minimum wages for cross-border workers.

Just as Swiss jurists exploited the French woman, it appears that Mathieu Parreaux has exploited Francois Thiébaud of Vevey as a puppet to exploit the reputation of the much larger-than-life Monsieur Thiébaud at Tissot & SWATCH Group.

12 October, 2025 02:00PM

Arnaud Parreaux lost case defending rogue employer

Publicity from Parreaux, Thiébaud & Partners claimed the firm works for the underdog. Cross border workers from France, the Frontaliers, are frequently victims of exploitation in Switzerland. The Swiss jurists at Parreaux, Thiébaud & Partners explicitly promised to help Frontaliers.

Searching records from the Swiss Federal Tribunal, we could only find one case where the name Parreaux is mentioned. The jurist for the case was Arnaud Parreaux, who was listed as a member of the team in Parreaux, Thiébaud & Partners / Justicia SA.

Reading the judgment, we see that an employer was penalized for underpayment of workers in Geneva. The employer decided to appeal the decision to the Swiss Federal Tribunal. Arnaud Parreaux was not representing the workers, he was representing the employer.

In other words, their promise to represent Frontaliers was just another lie.

Moreover, the rogue employer was an insurance sales office with five employees. The employer claimed the employees were really independent agents who were provided with desks where they could come and work for their clients. The canton and the tribunals felt that this arrangement was a sham and the employees had been underpaid by twenty thousand Swiss francs.

The Swiss Federal Tribunal was not convinced by the arguments of Arnaud Parreaux. He lost the case and the punishment from the cantonal tribunal was upheld by the Federal tribunal.

Search the judgments of the Swiss Federal Tribunal online.

In the search box, type the name "parreaux".

Swiss Federal Tribunal, Parreaux, search

 

The result is a single case: 20.03.2024 2C 137/2024: penalty for failure to respect minimum salary. (Amende pour non respect du salaire minimum).

2C_137/2024

Arrêt du 20 mars 2024

IIe Cour de droit public

Composition
Mmes et M. les Juges fédéraux
Aubry Girardin, Présidente,
Ryter et Kradolfer,
Greffier : M. Dubey.

Participants à la procédure
A.________ Sàrl,
représentée par Me Arnaud Parreaux, avocat,
recourante,

contre

Service cantonal de l'inspection et des relations du
travail de la République et du canton de Genève,
rue des Noirettes 35, 1227 Carouge GE,
intimé.

Objet
Amende pour non respect du salaire minimum,

recours contre l'arrêt de la Cour de Justice du canton de Genève (2e Section) du 30 janvier 2024 (ATA/117/2024).

Swiss Federal Tribunal, Lausanne

12 October, 2025 01:30PM

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

RcppSpdlog 0.0.23 on CRAN: New Upstream

Version 0.0.23 of RcppSpdlog arrived on CRAN today (after a slight delay) and has been uploaded to Debian. RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovich. You can learn more at the nice package documention site.

This release updates the code to the version 1.16.0 of spdlog which was released yesterday morning, and includes version 1.12.0 of fmt. We also converted the documentation site to now using mkdocs-material to altdoc (plus local style and production tweaks) rather than directly.

I updated the package yesterday morning when spdlog was updated. But the passage was delayed for a day at CRAN as their machines still times out hitting the GPL-2 URL from the README.md badge, leading to a human to manually check the log assert the nothingburgerness of it. This timeout does not happen to me locally using the corresponding URL checker package. I pondered this in a r-package-devel thread and may just have to switch to using the R Project URL for the GPL-2 as this is in fact recurrning.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.23 (2025-10-11)

  • Upgraded to upstream release spdlog 1.16.0 (including fmt 12.0)

  • The mkdocs-material documentation site is now generated via altdoc

Courtesy of my CRANberries, there is also a diffstat report detailing changes. More detailed information is on the RcppSpdlog page, or the package documention site.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

12 October, 2025 11:43AM

Ingo Juergensmann

Outages on Nerdculture.de due to Ceph

Well, maybe it’s not entirely correct to blame Ceph for outages that happened in the last weeks to Nerdculture.de and other services running on my servers, but, well, I need to start somehow…

Overview

Shortly after the update from Debian 12 “Bookworm” to Debian 13 “Trixie” I also updated the Debian-based Proxmox installations. And then the issues began and I had sleepless nights, many downtimes and frustrated users, because the usually rock-stable Ceph storage became unstable. The OSDs went off the net, the Ceph Filesystem got degraded and everything became slow. The Ceph Filesystems (CephFS) also holds the mail storage as well as the shared storage (code & data) for my Nerdculture.de Mastodon instance.

Just to outline what I’m about to discuss, here’s the cabling plan for my 3-node hyperconverged Proxmox server setup:

Basically you see 3 types of connections:
1) Internet connection to the colocation switch
2) Internal Proxmox connections between the 3 nodes
3) Internal Ceph connections between the 3 nodes

The internal, directly wired connections are necessary because the colocation provider have had no additional Copper 10 Gbit/s ports (10GbaseT) available. So I had to wire up all those by directly attached patch cables.

Ceph has a backend and frontend network. You can run Ceph with just one network, but well, then Proxmox and Ceph would need to share the same network and access to Ceph would slow down when Virtual Machines (VMs) were migrated between the nodes.

What happened the last weeks?

The problem started, as said, after updating the Proxmox nodes. On Sept. 24th the first outage happened. You can read my summary here. Somehow the network Ceph connection between didn’t work anymore. The setup that was running for years now didn’t work anymore. The Ceph backend network couldn’t see the disks (OSD) any longer, so added manual routes between the nodes instead of relying on FRR with OSPF (a dynamic routing protocol). This solved the problem back then.

The next issue happened a week later on Oct. 2nd: since the last issue a week before I discovered that CephFS was awkfully slow. Loading the mails took like 10 seconds instead of being instantly there. So I tried to find the reason. My best assumption was: the WD Red SA500 2 TB that are holding the WAL/DB for the Ceph cluster are reaching the wear level end. These SSDs are not made for that kind of workload.

Another reason might be that the Ceph frontend network, which uses the Proxmox network, because the VMs need to access the Ceph frontend, is a OpenVSwitch bridge and traffic from Baldur is using the link via Pepper to Gate, for example, instead of using the direct connection, which adds some latency and reducing bandwidth.

And with that being said, this was the reason why there was an outage yesterday as well:

For the Ceph backend network, I use an internal Linux bridge in Proxmox to hold the IP for the Ceph backend on each node. Then there are two network cards, as described in the drawing. On the link between the nodes I configured Point to Point connections and added a route for the direct neighbor with a lower metric and a route for the other node with a higher metric. The other link vice versa. This works pretty well for the Ceph backend.

Yesterday I wanted to deploy those changes as well to the Proxmox network and get rid off of that Layer 2 network via OpenVSwitch. Settings this up in the operating system was no big deal, but unfortunately Proxmox complained later about the nodes having more than one IP. And there the issue started again.

But there was another problem, because even when reverting that network change, the Ceph cluster had issues again and couldn’t find its peers. I restarted services, rebooted nodes, etc… whatever to make it work again. But still OSDs were failing, coming online again, and failing again. The service mnt-pve-cephfs.mount was not able to mount the CephFS and thus CephFS was not available for the VMs and therefore the services like mail and Mastodon failed to load as well as nearly all services that need SSL certificates which – you guess it! – lies on CephFS as well. No CephFS available, no SSL certs and no service.

But why was it not possible to mount the CephFS on the Proxmox host nodes? I had a look onto the syslog and other logs while restarting services, but the output was that much and fast, that I couldn’t find the root cause for it.

At one time I was lucky and spotted this line:
2025-10-12T01:05:28.209050+02:00 baldur ceph-mgr[40880]: ERROR:root:Module 'xmltodict' is not installed.
And the solution was as simple as searching the web for that error message and stumble across this post in the Proxmox forum:

I was able to correct this with python3-xmltodict, that resolved one issue
So, after installing that package the Ceph cluster was happy again and Proxmox could finally mount CephFS with restarting mnt-pve-cephfs.mount.

Then it was just a matter of restarting VMs and services and finally Mastodon on https://nerdculture.de/ was available again as well as mail started to come in.

Lessons learned

For one I’m going to buy new SSDs for WAL/DB in Ceph, most likely Micron 5400 MAX. This should bringt he latency down with Ceph and increase the overall speed, because data is only written for the client, when all 3 nodes have written their data to disk. The slowest node or disk is the resulting speed of Ceph. WD Red SSDs might be good enough for NAS systems, but for constant disk writes like in the case of WAL/DB in Ceph, they seem to hit their limit rather soon.

Another thing I could improve is the network. It is a complex setup and prone to errors. I need to talk to the colocation if I can get 6x 10 Gbps ports on their switch or if I can bring in my own switch and what that would cost?

Speaking of: what switch would you recommend?

12 October, 2025 10:50AM by ij

October 11, 2025

John Goerzen

A Mail Delivery Mystery: Exim, systemd, setuid, and Docker, oh my!

On mail.quux, a node of NNCPNET (the NNCP-based peer-to-peer email network), I started noticing emails not being delivered. They were all in the queue, frozen, and Exim’s log had entries like:

unable to set gid=5001 or uid=5001 (euid=100): local delivery to [redacted] transport=nncp

Weird.

Stranger still, when I manually ran the queue with sendmail -qff -v, they all delivered fine.

Huh.

Well, I thought, it was a one-off weird thing. But then it happened again.

Upon investigating, I observed that this issue was happening only on messages submitted by SMTP. Which, on these systems, aren’t that many.

While trying different things, I tried submitting a message to myself using SMTP. Nothing to do with NNCP at all. But look at this:

 jgoerzen@[redacted] R=userforward defer (-1): require_files: error for /home/jgoerzen/.forward: Permission denied

Strraaannnge….

All the information I could find about this, even a FAQ entry, said that the problem is that Exim isn’t setuid root. But it is:

-rwsr-xr-x 1 root root 1533496 Mar 29  2025 /usr/sbin/exim4

This problem started when I upgraded to Debian Trixie. So what changed there?

There are a lot of possibilities; this is running in Docker using my docker-debian-base system, which runs a regular Debian in Docker, including systemd.

I eventually tracked it down to Exim migrating from init.d to systemd in trixie, and putting a bunch of lockdowns in its service file. After a bunch of trial and error, I determined that I needed to override this set of lockdowns to make it work. These overrides did the trick:

ProtectClock=false
PrivateDevices=false
RestrictRealtime=false
ProtectKernelModules=false
ProtectKernelTunables=false
ProtectKernelLogs=false
ProtectHostname=false

I don’t know for sure if the issue is related to setuid. But if it is, there’s nothing that immediately jumps out at me about any of these that would indicate a problem with setuid.

I also don’t know if running in Docker makes any difference.

Anyhow, problem fixed, but mystery not solved!

11 October, 2025 01:44AM by John Goerzen

October 10, 2025

hackergotchi for Louis-Philippe Véronneau

Louis-Philippe Véronneau

Montreal's Debian & Stuff - September 2025

Our Debian User Group met on September 27th for our first meeting since our summer hiatus. As always, it was fun and productive!

Here's what we did:

pollo:

sergiodj:

LeLutin:

tvaz:

  • answered applicants (usual Application Manager stuff) as part of the New Member team
  • dealt with less pleasant stuff as part of the Community team
  • learned about aibohphobia!

viashimo:

  • looked at hardware on PCPartPicker
  • starting to port a zig version of soundscraper from zig 0.12 to 0.15.1

tassia:

Pictures

This time again, we were hosted at La Balise (formely ATSÉ).

It's nice to see this community project continuing to improve: the social housing apartments on the top floors should be opening this month! Lots of construction work was also ongoing to make the Espace des Possibles more accessible from the street level.

Group photo

Some of us ended up grabbing a drink after the event at l'Isle de Garde, a pub right next to the venue, but I didn't take any pictures.

10 October, 2025 11:16PM by Louis-Philippe Véronneau

Reproducible Builds

Reproducible Builds in September 2025

Welcome to the September 2025 report from the Reproducible Builds project!

Welcome to the very latest report from the Reproducible Builds project. Our monthly reports outline what we’ve been up to over the past month, and highlight items of news from elsewhere in the increasingly-important area of software supply-chain security. As ever, if you are interested in contributing to the Reproducible Builds project, please see the Contribute page on our website.

In this report:

  1. Reproducible Builds Summit 2025
  2. Can’t we have nice things?
  3. Distribution work
  4. Tool development
  5. Reproducibility testing framework
  6. Upstream patches

Reproducible Builds Summit 2025

Please join us at the upcoming Reproducible Builds Summit, set to take place from October 28th — 30th 2025 in Vienna, Austria!

We are thrilled to host the eighth edition of this exciting event, following the success of previous summits in various iconic locations around the world, including Venice, Marrakesh, Paris, Berlin, Hamburg and Athens. Our summits are a unique gathering that brings together attendees from diverse projects, united by a shared vision of advancing the Reproducible Builds effort.

During this enriching event, participants will have the opportunity to engage in discussions, establish connections and exchange ideas to drive progress in this vital field. Our aim is to create an inclusive space that fosters collaboration, innovation and problem-solving.

If you’re interesting in joining us this year, please make sure to read the event page which has more details about the event and location. Registration is open until 20th September 2025, and we are very much looking forward to seeing many readers of these reports there!


Can’t we have nice things?

Debian Developer Gunnar Wolf blogged that George V. Neville-Neil’s “Kode Vicious” column in Communications of the ACM in which reproducible builds “is mentioned without needing to introduce it (assuming familiarity across the computing industry and academia)”. Titled, Can’t we have nice things?, the article mentions:

Once the proper measurement points are known, we want to constrain the system such that what it does is simple enough to understand and easy to repeat. It is quite telling that the push for software that enables reproducible builds only really took off after an embarrassing widespread security issue ended up affecting the entire Internet. That there had already been 50 years of software development before anyone thought that introducing a few constraints might be a good idea is, well, let’s just say it generates many emotions, none of them happy, fuzzy ones. []


Distribution work

In Debian this month, Johannes Starosta filed a bug against the debian-repro-status package, reporting that it does not work on Debian trixie. (An upstream bug report was also filed.) Furthermore, 17 reviews of Debian packages were added, 10 were updated and 14 were removed this month adding to our knowledge about identified issues.

In March’s report, we included the news that Fedora would aim for 99% package reproducibility. This change has now been deferred to Fedora 44 according to Phoronix.

Lastly, Bernhard M. Wiedemann posted another openSUSE monthly update for their work there.


Tool development

diffoscope version 306 was uploaded to Debian unstable by Chris Lamb. It included contributions already covered in previous months as well as some changes by Zbigniew Jędrzejewski-Szmek to address issues with the fdtump support [] and to move away from the deprecated codes.open method. [][]

strip-nondeterminism version 1.15.0-1 was uploaded to Debian unstable by Chris Lamb. It included a contribution by Matwey Kornilov to add support for inline archive files for Erlang’s escript [].

kpcyrd has released a new version of rebuilderd. As a quick recap, rebuilderd is an automatic build scheduler that tracks binary packages available in a Linux distribution and attempts to compile the official binary packages from their (purported) source code and dependencies. The code for in-toto attestations has been reworked, and the instances now feature a new endpoint that can be queried to fetch the list of public-keys an instance currently identifies itself by. []

Lastly, Holger Levsen bumped the Standards-Version field of disorderfs, with no changes needed. [][]


Reproducibility testing framework

The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In August, however, a number of changes were made by Holger Levsen, including:

  • Setting up six new rebuilderd workers with 16 cores and 16 GB RAM each.

  • reproduce.debian.net-related:

    • Do not expose pending jobs; they are confusing without explaination. []
    • Add a link to v1 API specification. []
    • Drop rebuilderd-worker.conf on a node. []
    • Allow manual scheduling for any architectures. []
    • Update path to trixie graphs. []
    • Use the same rebuilder-debian.sh script for all hosts. []
    • Add all other suites to all other archs. [][][][]
    • Update SSH host keys for new hosts. []
    • Move to the pull184 branch. [][][][][]
    • Only allow 20 GB cache for workers. []
  • OpenWrt-related:

    • Grant developer aparcar full sudo control on the ionos30 node. [][]
  • Jenkins nodes:

    • Add a number of new nodes. [][][][][]
    • Dont expect /srv/workspace to exist on OSUOSL nodes. []
    • Stop hardcoding IP addresses in munin.conf. []
    • Add maintenance and health check jobs for new nodes. []
    • Document slight changes in IONOS resources usage. []
  • Misc:

    • Drop disabled Alpine Linux tests for good. []
    • Move Debian live builds and some other Debian builds to the ionos10 node. []
    • Cleanup some legacy support from releases before Debian trixie. []

In addition, Jochen Sprickerhof made the following changes relating to reproduce.debian.net:

  • Do not expose pending jobs on the main site. []
  • Switch the frontpage to reference Debian forky [], but do not attempt to build Debian forky on the armel architecture [].
  • Use consistent and up to date rebuilder-debian.sh script. []
  • Fix supported worker architectures. []
  • Add a basic ‘excuses’ page. []
  • Move to the pull184 branch. [][][][]
  • Fix a typo in the JavaScript. []
  • Update front page for the new v1 API. [][]

Lastly, Roland Clobus did some maintenance relating to the reproducibility testing of the Debian Live images. [][][][]


Upstream patches

The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:



Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

10 October, 2025 07:52PM

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

RcppArmadillo 15 CRAN Transition: Offering Office Hours

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1273 other packages on CRAN, downloaded 41.8 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 651 times according to Google Scholar.

Armadillo 15 brought changes. We mentioned these in the 15.0.2-1 and 15.0.2-1 release blog posts:

  • Minimum C++ standard of C++14
  • No more suppression of deprecation notes

(The second point is a consequence of the first. Prior to C++14, deprecation notes were issue via a macro, and the macro was set up by Conrad in the common way of allowing an override, which we took advantage of in RcppArmadillo effectively shielding downstream package. In C++14 this is now an attribute, and those cannot be suppressed.)

We tested this then-upcoming change extensively: Thirteen reverse dependency runs expoloring different settings and leading to the current package setup where an automatic fallback to the last Armadillo 14 release offers fallback for hardwired C++11 use and Armadillo 15 others. Given the 1200+ reverse deoendencies, this took considerable time. All this was also quite extensively discussed with CRAN (especially Kurt Hornik) and documented / controlled via a series of issue tickets starting with overall issue #475 covering the subissues:

  • open issue #475 describes the version selection between Armadillo 14 and 15 via #define
  • open issue #476 illustrates how package without deprecation notes are already suitable for Armadillo 15 and C++14
  • open issue #477 demonstrates how a package with a simple deprecation note can be adjusted for Armadillo 15 and C++14
  • closed issue #479 documents a small bug we created in the initial transition package RcppArmadillo 15.0.1-1 and fixed in the 15.0.2-1
  • closed issue #481 discusses removal of the check for insufficient LAPACK routines which has been removed given that R 4.5.0 or later has sufficient code in its fallback LAPACK (used e.g. on Windows)
  • open issue #484 offering help to the (then 226) packages needing help transitioning from (enforced) C++11
  • open issue #485 offering help to the (then 135) packages needing help with deprecations
  • open issue #489 coordinating pull requests and patches to 35 packages for the C++11 transition
  • open issue #491 coordinating pull requests and patches to 25 packages for deprecation transition

The sixty pull requests (or emailed patches) followed a suggestion by CRAN to rank-order packages affected by their reverse dependencies sorted in descending package count. Now, while this change from Armadillo 14 to 15 was happening, CRAN also tightened the C++11 requirement for packages and imposed a deadline for changes. In discussion, CRAN also convinced me that a deadline for the deprecation warning, now unmasked, was viable (and is in fairness commensurate with similar, earlier changes triggered by changes in the behaviour of either gcc/g++ or clang/clang++). So we now have two larger deadline campaigns affecting the package (and as always there are some others).

These deadlines are coming close: October 17 for the C++11 transition, and October 23 for the deprecation warning. Now, as became clear preparing the sixty pull requests and patches, these changes are often relatively straightforward. For the former, remove the C++11 enforcement and the package will likely build without changes. For the latter, make the often simple (e.g. swith from arma::is_finite to std::isfinite) change. I did not encounter anything much more complicated yet.

The number of affected packages—approximated by looking at all packages with a reverse dependency on RcppArmadillo and having a deadline–can be computed as

suppressMessages(library(data.table))
D <- setDT(tools::CRAN_package_db())
P <- data.table(Package=tools::package_dependencies("RcppArmadillo", reverse=TRUE, db=D)[[1]])
W <- merge(P, D, all.x=TRUE)[is.na(Deadline)==FALSE,c(1:2,38,68)]
W

W[, nrevdep := length(tools::package_dependencies(Package, reverse=TRUE, recursive=TRUE, db=D)[[1]]), by=Package]
W[order(-nrevdep)]

and has been declining steadily from over 350 to now under 200. For that a big and heartfelt Thank You! to all the maintainers who already addressed their package and uploaded updated packages to CRAN. That rocks, and is truly appreciated.

Yet the number is still large. And while issues #489 and #491 show a number of ‘pending’ packages that have merged but not uploaded (yet?) there are also all the other packages I have not been able to look at in detail. While preparing sixty PRs / patches was viable over a period of a good week, I cannot create these for all packages. So with that said, here is a different suggestion for help: All of next week, I will be holding open door ‘open source’ office hours online two times each day (11:00h to 13:00h Central, 16:00h to 18:00h Central) which can be booked via this booking link for Monday to Friday next week in either fifteen or thirty minutes slots you can book. This should offer Google Meet video conferencing (with jitsi as an alternate, you should be able to control that) which should allow for screen sharing. (I cannot hookup Zoom as my default account has organization settings with a different calendar integration.)

If you are reading this and have a package that still needs helps, I hope to see you in the Open Source Office Hours to aid in the RcppArmadillo package updates for your package. Please book a slot!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

10 October, 2025 02:20PM

John Goerzen

October 09, 2025

Thorsten Alteholz

My Debian Activities in September 2025

Debian LTS

This was my hundred-thirty-fifth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. During my allocated time I uploaded or worked on:

  • [DLA 4168-2] openafs regression update to fix an incomplete patch in the previous upload.
  • [DSA 5998-1] cups security update to fix two CVEs related to a authentication bypass and a denial of service.
  • [DLA 4298-1] cups security update to fix two CVEs related to a authentication bypass and a denial of service.
  • [DLA 4304-1] cjson security update to fix one CVE related to an out-of-bounds memory access.
  • [DLA 4307-1] jq security update to fix one CVE related to a heap buffer overflow.
  • [DLA 4308-1] corosync security update to fix one CVE related to a stack-based buffer overflow.

An upload of spim was not needed, as the corresponding CVE could be marked as ignored. I also started to work on an open-vm-tools and attended the monthly LTS/ELTS meeting.

Debian ELTS

This month was the eighty-sixth ELTS month. During my allocated time I uploaded or worked on:

  • [ELA-1512-1] cups security update to fix two CVEs in Buster and Stretch, related to a authentication bypass and a denial of service.
  • [ELA-1520-1] jq security update to fix one CVE in Buster and Stretch, related to a heap buffer overflow.
  • [ELA-1524-1] corosync security update to fix one CVE in Buster and Stretch, related to a stack-based buffer overflow.
  • [ELA-1527-1] mplayer security update to fix ten CVEs in Stretch, distributed all over the code.

The CVEs for open-vm-tools could be marked as not-affeceted as the corresponding plugin was not yet available. I also attended the monthly LTS/ELTS meeting.

Debian Printing

This month I uploaded a new upstream version or a bugfix version of:

  • ink to unstable to fix a gcc15 issue.
  • pnm2ppa to unstable to fix a gcc15 issue.
  • rlpr to unstable to fix a gcc15 issue.

This work is generously funded by Freexian!

Debian Astro

This month I uploaded a new upstream version or a bugfix version of:

Debian IoT

This month I uploaded a new upstream version or a bugfix version of:

  • radlib to unstable, Joachim Zobel prepared a patch for a name collision of a binary.
  • pyicloud to unstable.

Debian Mobcom

This month I uploaded a new upstream version or a bugfix version of:

misc

The main topic of this month has been gcc15 and cmake4, so my upload rate was extra high. This month I uploaded a new upstream version or a bugfix version of:

  • readsb to unstable.
  • gcal to unstable. This was my first upload of a release where I am upstream as well.
  • libcds to unstable to fix a cmake4 issue.
  • pkcs11-proxy to unstable to fix cmake4 issue.
  • force-ip-protocol to unstable to fix a gcc15 issue.
  • httperf to unstable to fix a gcc15 issue.
  • otpw to unstable to fix a gcc15 issue.
  • rplay to unstable to fix a gcc15 issue.
  • uucp to unstable to fix a gcc15 issue.
  • spim to unstable to fix a gcc15 issue.
  • usb-modeswitch to unstable to fix a gcc15 issue.
  • gnucobol3 to unstable to fix a gcc15 issue.
  • gnucobol4 to unstable to fix a gcc15 issue.

I wonder what MBF will happen next, I guess the /var/lock-issue will be a good candidate.

On my fight against outdated RFPs, I closed 30 of them in September. Meanwhile only 3397 are still open, so don’t hesitate to help closing one or another.

FTP master

This month I accepted 294 and rejected 28 packages. The overall number of packages that got accepted was 294.

09 October, 2025 02:24PM by alteholz

October 08, 2025

hackergotchi for Colin Watson

Colin Watson

Free software activity in September 2025

About 90% of my Debian contributions this month were sponsored by Freexian.

You can also support my work directly via Liberapay or GitHub Sponsors.

Some months I feel like I’m pedalling furiously just to keep everything in a roughly working state. This was one of those months.

Python team

I upgraded these packages to new upstream versions:

  • aiosmtplib
  • billiard
  • dbus-fast
  • django-modeltranslation
  • django-sass-processor
  • feedparser
  • flask-security
  • jaraco.itertools
  • mariadb-connector-python
  • mistune
  • more-itertools
  • pydantic-settings
  • pyina
  • pytest-mock
  • python-asyncssh
  • python-bytecode
  • python-ciso8601
  • python-django-pgbulk
  • python-ewokscore
  • python-ewoksdask
  • python-ewoksutils
  • python-expandvars
  • python-git
  • python-gssapi
  • python-holidays
  • python-jira
  • python-jpype
  • python-mastodon
  • python-orjson (fixing a build failure)
  • python-pyftpdlib
  • python-pytest-asyncio (fixing a build failure)
  • python-pytest-run-parallel
  • python-recurring-ical-events
  • python-redis
  • python-watchfiles (fixing a build failure)
  • python-x-wr-timezone
  • python-zipp
  • pyzmq
  • readability
  • scalene (fixing test failures with pydantic 2.12.0~a1)
  • sen (contributed supporting fix upstream)
  • sqlfluff
  • trove-classifiers
  • ttconv
  • vdirsyncer
  • zope.component
  • zope.configuration
  • zope.deferredimport
  • zope.deprecation
  • zope.exceptions
  • zope.i18nmessageid
  • zope.interface
  • zope.proxy
  • zope.schema
  • zope.security (contributed supporting fix upstream)
  • zope.testing
  • zope.testrunner

I had to spend a fair bit of time this month chasing down build/test regressions in various packages due to some other upgrades, particularly to pydantic, python-pytest-asyncio, and rust-pyo3:

After some upstream discussion I requested removal of pydantic-compat, since it was more trouble than it was worth to keep it working with the latest pydantic version.

I filed dh-python: pybuild-plugin-pyproject doesn’t know about headers and added it to Python/PybuildPluginPyproject, and converted some packages to pybuild-plugin-pyproject:

I updated dh-python to suppress generated dependencies that would be satisfied by python3 >= 3.11.

pkg_resources is deprecated. In most cases replacing it is a relatively simple matter of porting to importlib.resources, but packages that used its old namespace package support need more complicated work to port them to implicit namespace packages. We had quite a few bugs about this on zope.* packages, but fortunately upstream did the hard part of this recently. I went round and cleaned up most of the remaining loose ends, with some help from Alexandre Detiste. Some of these aren’t completely done yet as they’re awaiting new upstream releases:

This work also caused a couple of build regressions, which I fixed:

I fixed jupyter-client so that its autopkgtests would work in Debusine.

I fixed waitress to build with the nocheck profile.

I fixed several other build/test failures:

I fixed some other bugs:

Code reviews

Other bits and pieces

I fixed several CMake 4 build failures:

I got CI for debbugs passing (!22, !23).

I fixed a build failure with GCC 15 in trn4.

I filed a release-notes bug about the tzdata reorganization in the trixie cycle.

I filed and fixed a git-dpm regression with bash 5.3.

I upgraded libfilter-perl to a new upstream version.

I optimized some code in ubuntu-dev-tools that made O(1) HTTP requests when it could instead make O(n).

08 October, 2025 06:16PM by Colin Watson

Sven Hoexter

Backstage Render Markdown in a Collapsible Block

Brief note to maybe spare someone else the trouble. If you want to hide e.g. a huge table in Backstage (techdocs/mkdocs) behind a collapsible element you need the md_in_html extension and use the markdown attribute for it to kick in on the <details> html tag.

Add the extension to your mkdocs.yaml:

markdown_extensions:
  - md_in_html

Hide the table in your markdown document in a collapsible element like this:

<details markdown>
<summary>Long Table</summary>

| Foo | Bar |
|-|-|
| Fizz | Buzz |

</details>

It's also required to have an empty line between the html tag and starting the markdown part. Rendered for me that way in VSCode, GitHub and Backstage.

08 October, 2025 03:17PM

October 03, 2025

hackergotchi for Jonathan Dowland

Jonathan Dowland

Tron: Ares (soundtrack)

photo of Tron: Ares vinyl record on my turntable, next to packaging

There's a new Nine Inch Nails album! That doesn't happen very often. There's a new Trent Reznor & Atticus Ross soundtrack! That happens all the time! For the first time, they're the same thing.

The new one, Tron: Ares, is very deliberately presented as a Nine Inch Nails album, and not a TR&AR soundtrack. But is it neither fish nor fowl? 24 tracks, four with lyrics. Singing is not unheard of on TR&AR soundtracks, but it's rare (A Minute to Breathe from the excellent Before the Flood is another). Instrumentals are not rare on NIN albums, either, but this ratio is very unusual, and has disappointed some fans who were hoping for a more traditional NIN album.

What does it mean to label something a NIN album anyway? For me, the lines are now further blurred. One thing for sure is it means a lot of media attention, and this release, as well as the film it's promoting, are all over the media at the moment. Posters, trailers, promotional tie-in items, Disney logos everywhere. The album is hitched to the Disney marketing and promotion machine. It's a bit weird seeing the NIN logo all over the place advertising the movie.

On to the music. I love TR&AR soundtracks, and some of my favourite NIN tracks are instrumentals. Despite that, three highlights for me are songs: As Alive As You Need Me To Be, I Know You Can Feel It and closer Shadow Over Me. The other stand-out is Building Better Worlds, a short instrumental and clear nod to Wendy Carlos.

My main complaint here applies to some of the more recent soundtracks as well: the tracks are too short. They're scored to scenes in the movie, which makes a lot of sense in that presentation, but less so for independent listening. It's not a problem that their earlier, lauded soundtracks suffered (The Social Network, Before the Flood, Bird Box Extended). Perhaps a future remix album will address that.

03 October, 2025 10:01AM

hackergotchi for Guido Günther

Guido Günther

Free Software Activities September 2025

Another short status update of what happened on my side last month. Nothing stands out too much, I enjoyed doing the OSK changes the most as that helped to improve the typing experience further. Also doing a small bit of kernel work again was fun (still need to figure out the 6mq's touch controller repsonsiveness though).

See below for details on the above and more:

phosh

  • Add backlight brightness handling (MR)
  • Handle brightness keybinding (MR)
  • Use stevia (MR)
  • Test suite improvements (MR)
  • Simplify keybinding generation (MR)
  • Allow g-c-c to work against nested phosh (MR)
  • Hide demo plugins (MR)

phoc

  • Unbreak type to search (MR)
  • Update to wlroots 0.19.1 (MR)
  • Relese 0.50~rc1
  • Catch up with wlroots git (MR)
  • Damage tracking and render simplifications (MR)

phosh-mobile-settings

  • Allow to hide plugins (MR)
  • Release 0.50~rc1
  • Hide demo plugins by default (MR)
  • Sink floating refs properly (MR)
  • Simplify includes (MR)

stevia (formerly phosh-osk-stub)

  • Fix meson warning (MR)
  • Update URLs (MR)
  • Make backspace more clever (MR)
  • presage: Better handle predictions vs completions: (MR)

xdg-desktop-portal-phosh

  • Update to pfs 0.0.5 (MR)
  • Release 0.50~rc1
  • Allow to disable Rust portal (MR)
  • Use release ashpd (MR)

pfs

  • Release 0.0.5 (MR)

libphosh-rs

  • Modernize and release 0.0.7 (MR)

Phrog

  • Bump libphosh dependency to 0.0.7 (MR)

feedbackd

  • Release 0.8.5 (MR)
  • Publish API docs (MR)

feedbackd-device-themes

  • Release 0.8.6 (MR)

Debian

  • 0.46 backports for trixie: (MR) - testers needed!
  • cellbroadcastd: Upload to sid (MR)
  • meta-phosh: Update deps (MR)
  • meta-phosh: Adjust deps for 0.49 (MR)
  • phosh-tour: Upload to unstable (MR)
  • xdg-desktop-portal-phosh: Upload 0.50~rc1
  • xdg-desktop-portal-phosh: Enable Rust based portal (MR)
  • wlroots: Upload 0.19.1
  • rust-libphosh: Update to 0.0.7
  • Release Phosh 0.50~rc1
  • Release phosh-mobile-seettings 0.50~rc1
  • Release feedbackd 0.8.5
  • Release feedbackd-device-themes 0.8.6
  • Release phoc 0.50~rc1

gnome-settings-daemon

  • Fix brightness values (MR)

git-buildpackage

  • Make gbp import-orig --uscan useful again when passing in a version (MR)
  • Make dsc component tests fetch from salsa (MR)

govarnam

  • Fix gcc-15 build (MR)

Sessions

  • Fix missing application icon (MR)

twenty-twenty-hugo

  • Avoid 404 on each page load (MR)
  • Fingerprint custom CSS (MR)

tuwunnel

  • Fix alias in systemd unit (MR)
  • Document support items (MR)

Linux

  • Add backlight support for Shift6MQ (v1, v2, v3)

mutter

  • udev: Don't leak parent device (MR)

Phosh debs

  • Don't require gsd-49 yet (MR)

phosh-site

  • Fix links (MR)
  • Update several entries (MR)
  • Mention nonprofit (MR)
  • Automatic deploy (MR)

Reviews

This is not code by me but reviews I did on other peoples code. The list is (as usual) slightly incomplete. Thanks for the contributions!

  • p-m-s: Tweaks parsing (MR)
  • p-m-s: Prefer char over gchar (MR)
  • p-m-s/tweaks: Add .XResources backend (MR)
  • p-m-s/tweaks: Add Symlink backend (MR)
  • p-m-s/tweaks: Cleanup includes (MR)
  • p-m-s/tweaks: Cleanup self ref (MR)
  • p-m-s/tweaks: Menu toggle (MR)
  • p-m-s/tweaks: i18n support (MR)
  • p-m-s/tweaks: Use toasts for errors (MR)
  • p-m-s/run: Add gdb invocation (MR)
  • p-m-s: Appinfo tweaks (MR)
  • p-m-s: Hide Config tweaks menu entry when not needed (MR)
  • m-b-p-i provider updates: (MR)
  • m-b-p-i emergency number updates: (MR, MR, MR)
  • pfs: Switch to gtk-rs 0.10 (MR)
  • x-d-p-p: Switch to gtk-rs 0.10 (MR)
  • x-d-p-p: Port file chooser portal to Rust (MR)
  • phosh: custom lockscreen message (MR)
  • libcmatrix: Bump endpoint versions (MR)
  • phosh-recipes: Add gnome-software-plugin-flatpak (MR)

Help Development

If you want to support my work see donations.

Comments?

Join the Fediverse thread

03 October, 2025 08:08AM

October 02, 2025

John Goerzen

A Twisty Maze of Ill-Behaved Bots

Like many, bot traffic has been causing significant issues for my hosted server recently. I’ve been noticing a dramatic increase in bots that do not respect robots.txt, especially the crawl-delay I have set there. Not only that, but many of them are sending user-agent strings that are quite precisely matching what desktop browsers send. That is, they don’t identify themselves.

They posed a particular problem on two sites: my blog, and the lists.complete.org archives.

The list archives is a completely static site, but it has many pages, so the bots that are ill-behaved absolutely hammer it following links.

My blog runs WordPress. It has fewer pages, but by using PHP, doesn’t need as many hits to start to bog down. Also, there is a Mastodon thundering herd problem, and since I participate on Mastodon, this hits my server.

The solution was one of layers.

I had already added a crawl-delay line to robots.txt. It helped a bit, but many bots these days aren’t well-behaved. Next, I added WP Super Cache to my WordPress installation. I also enabled APCu in PHP and installed APCu Manager. Again, each step helped. Again, not quite enough.

Finally, I added Anubis. Installing it (especially if using the Docker container) was under-documented, but I figured it out. By default, it is designed to block AI bots and try to challenge everything with “Mozilla” in its user-agent (which is most things) with a Javascript challenge.

That’s not quite what I want. If a bot is well-behaved, AI or otherwise, it will respect my robots.txt and I can more precisely control it there. Also, I intentionally support non-Javascript browsers on many of the sites I host, so I wanted to be judicious. Eventually I configured Anubis to only challenge things that present a user-agent that looks fully like a real browser. In other words, real browsers should pass right through, and bad bots pretending to be real browsers will fail.

That was quite effective. It reduced load further to the point where things are ordinarily fairly snappy.

I had previously been using mod_security to block some bots, but it seemed to be getting in the way of the Fediverse at times. When I disabled it, I observed another increase in speed. Anubis was likely going to get rid of those annoying bots itself anyhow.

As a final step, I migrated to a faster hosting option. This post will show me how well it survives the Mastodon thundering herd!

Update: Yes, it handled it quite nicely now.

02 October, 2025 03:01AM by John Goerzen

October 01, 2025

hackergotchi for Ben Hutchings

Ben Hutchings

FOSS activity in September 2025

Last month I attended and spoke at Kangrejos, for which I will post a separate report later. Besides that, here’s the usual categorised list of work:

01 October, 2025 03:24PM by Ben Hutchings

Birger Schacht

Status update, September 2025

Regarding Debian packaging this was a rather quiet month. I uploaded version 1.24.0-1 of foot and version 2.8.0-1 of git-quick-stats. I took the opportunity and started migrating my packages to the new version 5 watch file format, which I think is much more readable than the previous format.

I also uploaded version 0.1.1-1 of libscfg to NEW. libscfg is a C implementation of the scfg configuration file format and it is a dependency of recent version of kanshi. kanshi is a tool similar to autorandr which allows you define output profiles and kanshi switches to the correct output profile on hotplug events. Once libscfg is in unstable I can finally update kanshi to the latest version.

A lot of time this month in finalizing a redesign of the output rendering of carl. carl is a small rust program I wrote that provides a calendar view similar to cal, but it comes with colors and ical file integration. That means that you can not only display a simple calendar, but also colorize/highlight dates based on various attributes or based on events on that day. In the initial versions of carl the output rendering was simply hardcoded into the app.

Screenshot of carl

This was a bit cumbersome to maintain and not configurable for users. I am using templating languages on a daily basis, so I decided I would reimplement the output generation of carl to use templates. I chose the minijinja Rust library which is “based on the syntax and behavior of the Jinja2 template engine for Python”. There are others out there, like tera, but minijinja seems to be more active in development currently. I worked on this implementation on and off for the last year and finally had the time to finish it up and write some additional tests for the outputs. It is easier to maintain templates than Rust code that uses write!() to format the output. I also implemented a configuration option for users to override the templates.

Additional to the output refactoring I also fixed couple of bugs and finally released v0.4.0 of carl.

In my dayjob I released version 0.53 of apis-core-rdf which contains the place lookup field which I implemented in August. A couple of weeks later we released version 0.54 which comes with a middleware to show pass on messages from the Django messages framework via response header to HTMX to trigger message popups. This implementation is based on the blog post Using the Django messages framework with HTMX. Version 0.55 was the last release in September. It contained preparations for refactoring the import logic as well as a couple of UX improvements.

01 October, 2025 05:28AM

September 30, 2025

hackergotchi for Junichi Uekawa

Junichi Uekawa

Start of fourth quarter of the year.

Start of fourth quarter of the year. The end of year is feeling close!

30 September, 2025 11:25PM by Junichi Uekawa

hackergotchi for Jonathan McDowell

Jonathan McDowell

Local Voice Assistant Step 5: Remote Satellite

The last (software) piece of sorting out a local voice assistant is tying the openWakeWord piece to a local microphone + speaker, and thus back to Home Assistant. For that we use wyoming-satellite.

I’ve packaged that up - https://salsa.debian.org/noodles/wyoming-satellite - and then to run I do something like:

$ wyoming-satellite --name 'Living Room Satellite' \
    --uri 'tcp://[::]:10700' \
    --mic-command 'arecord -r 16000 -c 1 -f S16_LE -t raw -D plughw:CARD=CameraB409241,DEV=0' \
    --snd-command 'aplay -D plughw:CARD=UACDemoV10,DEV=0 -r 22050 -c 1 -f S16_LE -t raw' \
    --wake-uri tcp://[::1]:10400/ \
    --debug

That starts us listening for connections from Home Assistant on port 10700, uses the openWakeWord instance on localhost port 10400, uses aplay/arecord to talk to the local microphone and speaker, and gives us some debug output so we can see what’s going on.

And it turns out we need the debug. This setup is a bit too flaky for it to have ended up in regular use in our household. I’ve had some problems with reliable audio setup; you’ll note the Python is calling out to other tooling to grab audio, which feels a bit clunky to me and I don’t think is the actual problem, but the main audio for this host is hooked up to the TV (it’s a media box), so the setup for the voice assistant needs to be entirely separate. That means not plugging into Pipewire or similar, and instead giving direct access to wyoming-satellite. And sometimes having to deal with how to make the mixer happy + non-muted manually.

I’ve also had some issues with the USB microphone + speaker; I suspect a powered USB hub would help, and that’s on my list to try out.

When it does work I have sometimes found it necessary to speak more slowly, or enunciate my words more clearly. That’s probably something I could improve by switching from the base.en to small.en whisper.cpp model, but I’m waiting until I sort out the audio hardware issue before poking more.

Finally, the wake word detection is a little bit sensitive sometimes, as I mentioned in the previous post. To be honest I think it’s possible to deal with that, if I got the rest of the pieces working smoothly.

This has ended up sounding like a more negative post than I meant it to. Part of the issue in a resolution is finding enough free time to poke things (especially as it involves taking over the living room and saying “Hey Jarvis” a lot), part of it is no doubt my desire to actually hook up the pieces myself and understand what’s going on. Stay tuned and see if I ever manage to resolve it all!

30 September, 2025 06:23PM

Antoine Beaupré

Proper services

During 2025-03-21-another-home-outage, I reflected upon what's a properly ran service and blurted out what turned out to be something important I want to outline more. So here it is, again, on its own for my own future reference.

Typically, I tend to think of a properly functioning service as having four things:

  1. backups
  2. documentation
  3. monitoring
  4. automation
  5. high availability (HA)

Yes, I miscounted. This is why you need high availability.

A service doesn't properly exist if it doesn't at least have the first 3 of those. It will be harder to maintain without automation, and inevitably suffer prolonged outages without HA.

The five components of a proper service

Backups

Duh. If data is maliciously or accidentally destroyed, you need a copy somewhere. Preferably in a way that malicious Joe can't get to.

This is harder than you think.

Documentation

I have an entire template for this. Essentially, it boils down to using https://diataxis.fr/ and this "audit" guide. For me, the most important parts are:

  • disaster recovery (includes backups, probably)
  • playbook
  • install/upgrade procedures (see automation)

You probably know this is hard, and this is why you're not doing it. Do it anyways, you'll think it sucks, it will grow out of sync with reality, but you'll be really grateful for whatever scraps you wrote when you're in trouble.

Any docs, in other words, is better than no docs, but are no excuse for doing the work correctly.

Monitoring

If you don't have monitoring, you'll know it fails too late, and you won't know it recovers. Consider high availability, work hard to reduce noise, and don't have machine wake people up, that's literally torture and is against the Geneva convention.

Consider predictive algorithm to prevent failures, like "add storage within 2 weeks before this disk fills up".

This is also harder than you think.

Automation

Make it easy to redeploy the service elsewhere.

Yes, I know you have backups. That is not enough: that typically restores data and while it can also include configuration, you're going to need to change things when you restore, which is what automation (or call it "configuration management" if you will) will do for you anyways.

This also means you can do unit tests on your configuration, otherwise you're building legacy.

This is probably as hard as you think.

High availability

Make it not fail when one part goes down.

Eliminate single points of failures.

This is easier than you think, except for storage and DNS ("naming things" not "HA DNS", that is easy), which, I guess, means it's harder than you think too.

Assessment

In the above 5 items, I currently check two in my lab:

  1. backups
  2. documentation

And barely: I'm not happy about the offsite backups, and my documentation is much better at work than at home (and even there, I have a 15 year backlog to catchup on).

I barely have monitoring: Prometheus is scraping parts of the infra, but I don't have any sort of alerting -- by which I don't mean "electrocute myself when something goes wrong", I mean "there's a set of thresholds and conditions that define an outage and I can look at it".

Automation is wildly incomplete. My home server is a random collection of old experiments and technologies, ranging from Apache with Perl and CGI scripts to Docker containers running Golang applications. Most of it is not Puppetized (but the ratio is growing). Puppet itself introduces a huge attack vector with kind of catastrophic lateral movement if the Puppet server gets compromised.

And, fundamentally, I am not sure I can provide high availability in the lab. I'm just this one guy running my home network, and I'm growing older. I'm thinking more about winding things down than building things now, and that's just really sad, because I feel we're losing (well that escalated quickly).

Side note about Tor

The above applies to my personal home lab, not work!

At work, of course, it's another (much better) story:

  1. all services have backups
  2. lots of services are well documented, but not all
  3. most services have at least basic monitoring
  4. most services are Puppetized, but not crucial parts (DNS, LDAP, Puppet itself), and there are important chunks of legacy coupling between various services that make the whole system brittle
  5. most websites, DNS and large parts of email are highly available, but key services like the the Forum, GitLab and similar applications are not HA, although most services run under replicated VMs that can trivially survive a total, single-node hardware failure (through Ganeti and DRBD)

30 September, 2025 03:00PM

Minor outage at Teksavvy business

This morning, internet was down at home. The last time I had such an issue was in February 2023, when my provider was Oricom. Now I'm with a business service at Teksavvy Internet (TSI), in which I pay 100$ per month for a 250/50 mbps business package, with a static IP address, on which I run, well, everything: email services, this website, etc.

Mitigation

Email

The main problem when the service goes down like this for prolonged outages is email. Mail is pretty resilient to failures like this but after some delay (which varies according to the other end), mail starts to drop. I am actually not sure what the various settings are among different providers, but I would assume mail is typically kept for about 24h, so that's our mark.

Last time, I setup VMs at Linode and Digital Ocean to deal better with this. I have actually kept those VMs running as DNS servers until now, so that part is already done.

I had fantasized about Puppetizing the mail server configuration so that I could quickly spin up mail exchangers on those machines. But now I am realizing that my Puppet server is one of the service that's down, so this would not work, at least not unless the manifests can be applied without a Puppet server (say with puppet apply).

Thankfully, my colleague groente did amazing work to refactor our Postfix configuration in Puppet at Tor, and that gave me the motivation to reproduce the setup in the lab. So I have finally Puppetized part of my mail setup at home. That used to be hand-crafted experimental stuff documented in a couple of pages in this wiki, but is now being deployed by Puppet.

It's not complete yet: spam filtering (including DKIM checks and graylisting) are not implemented yet, but that's the next step, presumably to do during the next outage. The setup should be deployable with puppet apply, however, and I have refined that mechanism a little bit, with the run script.

Heck, it's not even deployed yet. But the hard part / grunt work is done.

Other

The outage was "short" enough (5 hours) that I didn't take time to deploy the other mitigations I had deployed in the previous incident.

But I'm starting to seriously consider deploying a web (and caching) reverse proxy so that I endure such problems more gracefully.

Side note on proper services

Well that was dumb. I wrote this clever piece on what's a properly ran service and originally shoved it deep inside this service note instead of making a blog article.

That is now fixed, see 2025-09-30-proper-services instead.

Resolution

In the end, I didn't need any mitigation and the problem fixed itself. I did do quite a bit of cleanup so that feels somewhat good, although I despaired quite a bit at the amount of technical debt I've accumulated in the lab.

Timeline

Times are in UTC-4.

  • 6:52: IRC bouncer goes offline
  • 9:20: called TSI support, waited on the line 15 minutes then was told I'd get a call back
  • 9:54: outage apparently detected by TSI
  • 11:00: no response, tried calling back support again
  • 11:10: confirmed bonding router outage, no official ETA but "today", source of the 9:54 timestamp above
  • 12:08: TPA monitoring notices service restored
  • 12:34: call back from TSI; service restored, problem was with the "bonder" configuration on their end, which was "fighting between Montréal and Toronto"

30 September, 2025 02:59PM

Russell Coker

Russ Allbery

Review: Deep Black

Review: Deep Black, by Miles Cameron

Series: Arcana Imperii #2
Publisher: Gollancz
Copyright: 2024
ISBN: 1-3996-1506-8
Format: Kindle
Pages: 509

Deep Black is a far-future science fiction novel and the direct sequel to Artifact Space. You do not want to start here. I regretted not reading the novels closer together and had to refresh my memory of what happened in the first book.

The shorter fiction in Beyond the Fringe takes place between the two series novels and leads into some of the events in this book, although reading it is optional.

Artifact Space left Marca Nbaro at the farthest point of the voyage of the Greatship Athens, an unexpected heroine and now well-integrated into the crew. On a merchant ship, however, there's always more work to be done after a heroic performance. Deep Black opens with that work: repairs from the events of the first book, the never-ending litany of tasks required to keep the ship running smoothly, and of course the trade with aliens that drew them so far out into the Deep Black.

We knew early in the first book that this wouldn't be the simple, if long, trading voyage that most of the crew of the Athens was expecting, but now they have to worry about an unsettling second group of aliens on top of a potential major war between human factions. They don't yet have the cargo they came for, they have to reconstruct their trading post, and they're a very long way from home. Marca also knows, at this point in the story, that this voyage had additional goals from the start. She will slowly gain a more complete picture of those goals during this novel.

Artifact Space was built around one of the most satisfying plots in military science fiction (at least to me): a protagonist who benefits immensely from the leveling effect and institutional inclusiveness of the military slowly discovering that, when working at its best, the military can be a true meritocracy. (The merchant marine of the Athens is not military, precisely, since it's modeled on the trading ships of Venice, but it's close enough for the purposes of this plot.) That's not a plot that lasts into a sequel, though, so Cameron had to find a new spine for the second half of the story. He chose first contact (of a sort) and space battle.

The space battle parts are fine. I read a ton of children's World War II military fiction when I was a boy, and I always preferred the naval battles to the land battles. This part of Deep Black reminded me of those naval battles, particularly a book whose title escapes me about the Arctic convoys to the Soviet Union. I'm more interested in character than military adventure these days, but every once in a while I enjoy reading about a good space battle. This was not an exemplary specimen of the genre, but it delivered on all the required elements.

The first contact part was more original, in part because Cameron chose an interesting medium ground between total incomprehensibility and universal translators. He stuck with the frustrations of communication for considerably longer than most SF authors are willing to write, and it worked for me. This is the first book I've read in a while where superficial alien fluency with the mere words of a human language masks continuing profound mutual incomprehension. The communication difficulties are neither malicious nor a setup for catastrophic misunderstanding, but an intrinsic part of learning about a truly alien species. I liked this, even though it makes for slower and more frustrating progress. It felt more believable than a lot of first contact, and it forced the characters to take risks and act on hunches and then live with the consequences.

One of the other things that Cameron does well is maintain the steady rhythm of life on a working ship as a background anchor to the story. I've read a lot of science fiction that shows the day-to-day routine only until something more interesting and plot-focused starts happening and then seems to forget about it entirely. Not here. Marca goes through intense and adrenaline-filled moments requiring risk and fast reactions, and then has to handle promotion write-ups, routine watches, and studying for advancement. Cameron knows that real battles involve long periods of stressful waiting and incorporates them into the book without making them too boring, which requires a lot of writing skill.

I prefer the emotional magic of finding a place where one belongs, so I was not as taken with Deep Black as I was with Artifact Space, but that's the inevitable result of plot progression and not really a problem with this book. Marca is absurdly central to the story in ways that have a whiff of "chosen one" dynamics, but if one can suspend one's disbelief about that, the rest of the book is solid. This is, fundamentally, a book about large space battles, so save it when you're in the mood for that sort of story, but it was a satisfying continuation of the series. I will definitely keep reading.

Recommended if you enjoyed Artifact Space. If you didn't, Deep Black isn't going to change your mind.

Followed by Whalesong, which is not yet released (and is currently in some sort of limbo for pre-orders in the US, which I hope will clear up).

Rating: 7 out of 10

30 September, 2025 04:12AM

September 29, 2025

hackergotchi for Thomas Lange

Thomas Lange

Updates on FAIme service: Linux Mint 22.2 and trixie backports available

The FAIme service [1] now offers to build customized installation images for Xfce edition of Linux Mint 22.2 'Zara'.

For Debian 13 installations, you can select the kernel from backports for the trixie release, which is currently version 6.16. This will support newer hardware.

29 September, 2025 11:15AM

Russ Allbery

Review: The Incandescent

Review: The Incandescent, by Emily Tesh

Publisher: Tor
Copyright: 2025
ISBN: 1-250-83502-X
Format: Kindle
Pages: 417

The Incandescent is a stand-alone magical boarding school fantasy.

Your students forgot you. It was natural for them to forget you. You were a brief cameo in their lives, a walk-on character from the prologue. For every sentimental my teacher changed my life story you heard, there were dozens of my teacher made me moderately bored a few times a week and then I got through the year and moved on with my life and never thought about them again.

They forgot you. But you did not forget them.

Doctor Saffy Walden is Director of Magic at Chetwood, an elite boarding school for prospective British magicians. She has a collection of impressive degrees in academic magic, a specialization in demonic invocation, and a history of vague but lucrative government job offers that go with that specialty. She turned them down to be a teacher, and although she's now in a mostly administrative position, she's a good teacher, with the usual crop of promising, lazy, irritating, and nervous students.

As the story opens, Walden's primary problem is Nikki Conway. Or, rather, Walden's primary problem is protecting Nikki Conway from the Marshals, and the infuriating Laura Kenning in particular.

When Nikki was seven, she summoned a demon who killed her entire family and left her a ward of the school. To Laura Kenning, that makes her a risk who should ideally be kept far away from invocation. To Walden, that makes Nikki a prodigious natural talent who is developing into a brilliant student and who needs careful, professional training before she's tempted into trying to learn on her own.

Most novels with this setup would become Nikki's story. This one does not. The Incandescent is Walden's story.

There have been a lot of young-adult magical boarding school novels since Harry Potter became a mass phenomenon, but most of them focus on the students and the inevitable coming-of-age story. This is a story about the teachers: the paperwork, the faculty meetings, the funding challenges, the students who repeat in endless variations, and the frustrations and joys of attempting to grab the interest of a young mind. It's also about the temptation of higher-paying, higher-status, and less ethical work, which however firmly dismissed still nibbles around the edges.

Even if you didn't know Emily Tesh is herself a teacher, you would guess that before you get far into this novel. There is a vividness and a depth of characterization that comes from being deeply immersed in the nuance and tedium of the life that your characters are living. Walden's exasperated fondness for her students was the emotional backbone of this book for me. She likes teenagers without idealizing the process of being a teenager, which I think is harder to pull off in a novel than it sounds.

It was hard to quantify the difference between a merely very intelligent student and a brilliant one. It didn't show up in a list of exam results. Sometimes, in fact, brilliance could be a disadvantage — when all you needed to do was neatly jump the hoop of an examiner's grading rubric without ever asking why. It was the teachers who knew, the teachers who could feel the difference. A few times in your career, you would have the privilege of teaching someone truly remarkable; someone who was hard work to teach because they made you work harder, who asked you questions that had never occurred to you before, who stretched you to the very edge of your own abilities. If you were lucky — as Walden, this time, had been lucky — your remarkable student's chief interest was in your discipline: and then you could have the extraordinary, humbling experience of teaching a child whom you knew would one day totally surpass you.

I also loved the world-building, and I say this as someone who is generally not a fan of demons. The demons themselves are a bit of a disappointment and mostly hew to one of the stock demon conventions, but the rest of the magic system is deep enough to have practitioners who approach it from different angles and meaty enough to have some satisfying layered complexity. This is magic, not magical science, so don't expect a fully fleshed-out set of laws, but the magical system felt substantial and satisfying to me.

Tesh's first novel, Some Desperate Glory, was by far my favorite science fiction novel of 2023. This is a much different book, which says good things about Tesh's range and the potential of her work yet to come: adult rather than YA, fantasy rather than science fiction, restrained and subtle in places where Some Desperate Glory was forceful and pointed. One thing the books do have in common, though, is some structure, particularly the false climax near the midpoint of the book. I like the feeling of uncertainty and possibility that gives both books, but in the case of The Incandescent, I was not quite in the mood for the second half of the story.

My problem with this book is more of a reader preference than an objective critique: I was in the mood for a story about a confident, capable protagonist who was being underestimated, and Tesh was writing a novel with a more complicated and fraught emotional arc. (I'm being intentionally vague to avoid spoilers.) There's nothing wrong with the story that Tesh wanted to tell, and I admire the skill with which she did it, but I got a tight feeling in my stomach when I realized where she was going. There is a satisfying ending, and I'm still very happy I read this book, but be warned that this might not be the novel to read if you're in the mood for a purer competence porn experience.

Recommended, and I am once again eagerly awaiting the next thing Emily Tesh writes (and reminding myself to go back and read her novellas).

Content warnings: Grievous physical harm, mind control, and some body horror.

Rating: 8 out of 10

29 September, 2025 04:45AM

September 28, 2025

Review: Echoes of the Imperium

Review: Echoes of the Imperium, by Nicholas & Olivia Atwater

Series: Tales of the Iron Rose #1
Publisher: Starwatch Press
Copyright: 2024
ISBN: 1-998257-04-5
Format: Kindle
Pages: 547

Echoes of the Imperium is a steampunk fantasy adventure novel, the first of a projected series. There is another novella in the series, A Matter of Execution, that takes place chronologically before this novel, but which I am told that you should read afterwards. (I have not yet read it.) If Olivia Atwater's name sounds familiar, it's probably for the romantic fantasy Half a Soul. Nicholas Atwater is her husband.

William Blair, a goblin, was a child sailor on the airship HMS Caliban during the final battle that ended the Imperium, and an eyewitness to the destruction of the capital. Like every imperial solider, that loss made him an Oathbreaker; the fae Oath that he swore to defend the Imperium did not care that nothing a twelve-year-old boy could have done would have changed the result of the battle. He failed to kill himself with most of the rest of the crew, and thus was taken captive by the Coalition.

Twenty years later, William Blair is the goblin captain of the airship Iron Rose. It's an independent transport ship that takes various somewhat-dodgy contracts and has to avoid or fight through pirates. The crew comes from both sides of the war and has built their own working truce. Blair himself is a somewhat manic but earnest captain who doesn't entirely believe he deserves that role, one who tends more towards wildly risky plans and improvisation than considered and sober decisions. The rest of the crew are the sort of wild mix of larger-than-life personality quirks that populate swashbuckling adventure books but leave me dubious that stuffing that many high-maintenance people into one ship would go as well as it does.

I did appreciate the gunnery knitting circle, though.

Echoes of the Imperium is told in the first person from Blair's perspective in two timelines. One follows Blair in the immediate aftermath of the war, tracing his path to becoming an airship captain and meeting some of the people who will later be part of his crew. The other is the current timeline, in which Blair gets deeper and deeper into danger by accepting a risky contract with unexpected complications.

Neither of these timelines are in any great hurry to arrive at some destination, and that's the largest problem with this book. Echoes of the Imperium is long, sprawling, and unwilling to get anywhere near any sort of a point until the reader is deeply familiar with the horrific aftermath of the war, the mountains guilt and trauma many of the characters carry around, and Blair's impostor syndrome and feelings of inadequacy. For the first half of this book, I was so bored. I almost bailed out; only a few flashes of interesting character interactions and hints of world-building helped me drag myself through all of the tedious setup.

What saves this book is that the world-building is a delight. Once the characters finally started engaging with it in earnest, I could not put it down. Present-time Blair is no longer an Oathbreaker because he was forgiven by a fairy; this will become important later. The sites of great battles are haunted by ghostly echoes of the last moments of the lives of those who died (hence the title); this will become very important later. Blair has a policy of asking no questions about people's pasts if they're willing to commit to working with the rest of the crew; this, also, will become important later. All of these tidbits the authors drop into the story and then ignore for hundreds of pages do have a payoff if you're willing to wait for it.

As the reader (too) slowly discovers, the Atwaters' world is set in a war of containment by light fae against dark fae. Instead of being inscrutable and separate, the fae use humans and human empires as tools in that war. The fallen Imperium was a bastion of fae defense, and the war that led to the fall of that Imperium was triggered by the price its citizens paid for that defense, one that the fae could not possibly care less about. The creatures may be out of epic fantasy and the technology from the imagined future of Victorian steampunk, but the politics are that of the Cold War and containment strategies. This book has a lot to say about colonialism and empire, but it says those things subtly and from a fantasy slant, in a world with magical Oaths and direct contact with powers that are both far beyond the capabilities of the main characters and woefully deficient in in humanity and empathy. It has a bit of the feel of Greek mythology if the gods believed in an icy realpolitik rather than embodying the excesses of human emotion.

The second half of this book was fantastic. The found-family vibe among a crew of high-maintenance misfits that completely failed to cohere for me in the first half of the book, while Blair was wallowing in his feelings and none of the events seemed to matter, came together brilliantly as soon as the crew had a real problem and some meaty world-building and plot to sink their teeth into. There is a delightfully competent teenager, some satisfying competence porn that Blair finally stops undermining, and a sharp political conflict that felt emotionally satisfying, if perhaps not that intellectually profound. In short, it turns into the fun, adventurous romp of larger-than-life characters that the setting promises. Even the somewhat predictable mid-book reveal worked for me, in part because the emotions of the characters around that reveal sold its impact.

If you're going to write a book with a bad half and a good half, it's always better to put the good half second. I came away with very positive feelings about Echoes of the Imperium and a tentative willingness to watch for the sequel. (It reaches a fairly satisfying conclusion, but there are a lot of unresolved plot hooks.) I'm a bit hesitant to recommend it, though, because the first half was not very fun. I want to say that about 75% of the first half of the book could have been cut and the book would have been stronger for it. I'm not completely sure I'm right, since the Atwaters were laying the groundwork for a lot of payoff, but I wish that groundwork hadn't been as much of a slog.

Tentatively recommended, particularly if you're in the mood for steampunk fae mythology, but know that this book requires some investment.

Technically, A Matter of Execution comes first, but I plan to read it as a sequel.

Rating: 8 out of 10

28 September, 2025 04:32AM

September 27, 2025

hackergotchi for Bits from Debian

Bits from Debian

New Debian Developers and Maintainers (July and August 2025)

The following contributors got their Debian Developer accounts in the last two months:

  • Francesco Ballarin (ballarin)
  • Roland Clobus (rclobus)
  • Antoine Le Gonidec (vv221)
  • Guilherme Puida Moreira (puida)
  • NoisyCoil (noisycoil)
  • Akash Santhosh (akash)
  • Lena Voytek (lena)

The following contributors were added as Debian Maintainers in the last two months:

  • Andrew James Bower
  • Kirill Rekhov
  • Alexandre Viard
  • Manuel Traut
  • Harald Dunkel

Congratulations!

27 September, 2025 04:00PM by Jean-Pierre Giraud

Julian Andres Klode

Dependency Tries

As I was shopping groceries I had a shocking realization: The active dependencies of packages in a solver actually form a trie (a dependency A|B - “A or B” - of a package X is considered active if we marked X for install).

Consider the dependencies A|B|C, A|B, B|X. In most package managers these just express alternatives, that is, the “or” relationship, but in Debian packages, it also expresses a preference relationship between its operands, so in A|B|C, A is preferred over B and B over C (and A transitively over C).

This means that we can convert the three dependencies into a trie as follows:

Dependency trie of the three dependencies

Solving the dependency here becomes a matter of trying to install the package referenced by the first edge of the root, and seeing if that sticks. In this case, that would be ‘a’. Let’s assume that ‘a’ failed to install, the next step is to remove the empty node of a, and merging its children into the root.

Reduced dependency trie with “not A” containing b, b|c, b|x

For ease of visualisation, we remove “a” from the dependency nodes as well, leading us to a trie of the dependencies “b”, “b|c”, and “b|x”.

Presenting the Debian dependency problem, or the positive part of it as a trie allows us for a great visualization of the problem but it may not proof to be an effective implementation choice.

In the real world we may actually store this as a priority queue that we can delete from. Since we don’t actually want to delete from the queue for real, our queue items are pairs of a pointer to dependency and an activitity level, say A|B@1. Whenever a variable is assigned false, we look at its reverse dependencies and bump their activity, and reinsert them (the priority of the item being determined by the leftmost solution still possible, it has now changed). When we iterate the queue, we remove items with a lower activity level:

  1. Our queue is A|B@1, A|B|C@1, B|X@1
  2. Rejecting A bump the activity for its reverse dependencies and reinset them: Our queue is A|B@1, A|B|C@1, (A|)B@2, (A|)B|C@2, B|X@1
  3. We visit A|B@1 but see the activity of the underlying dependency is now 2 and remove it Our queue is A|B|C@1, (A|)B@2, (A|)B|C@2, B|X@1
  4. We visit A|B|C@1 but see the activity of the underlying dependency is now 2 and remove it Our queue is (A|)B@2, (A|)B|C@2, B|X@1
  5. We visit A|B@2, see the activity matches and find B is the solution.

27 September, 2025 02:32PM

September 25, 2025

hackergotchi for Steinar H. Gunderson

Steinar H. Gunderson

Negative result: Branch-free sparse bitset iteration

Sometimes, it's nice to write up something that was a solution to an interesting problem but that didn't work; perhaps someone else can figure out a crucial missing piece, or perhaps their needs are subtly different. Or perhaps they'll just find it interesting. This is such a post.

The problem in question is that I have a medium-sized sparse bitset (say, 1024 bits) and some of those bits (say, 20–50, but may be more and may be less) are set. I want to iterate over those bits, spending as little time as possible on the rest.

The standard formulation (as far as I know, anyway?), given modern CPUs, is to treat them as a series of 64-bit unsigned integers, and then use a double for loop like this (C++20, but should be easily adaptable to any low-level enough language):

// Assume we have: uint64_t arr[1024 / 64];

for (unsigned block = 0; block < 1024 / 64; ++block) {
   for (unsigned bits = arr[block]; bits != 0; bits &= bits - 1) {
       unsigned idx = 64 * block + std::countr_zero(bits);
       // do something with idx here
   }
}

The building blocks are simple enough if you're familiar with bit manipulation; std::countr_zero() invokes a bit-scan instruction, and

bits &= bits - 1
clears the lowest set bit.

This is roughly proportional to the number of set bits in the bit set, except that if you have lots of zeros, you'll spend time skipping over empty blocks. That's fine. What's not fine is that this is a disaster for the branch predictor, and my code was (is!) spending something like 20% of its time in the CPU handling mispredicted branches. The structure of the two loops is just so irregular; what we'd like is a branch-free way of iterating.

Now, we can of course never be fully branch-free; in particular, we need to end the loop at some point, and that branch needs to be predicted. So call it branch…less? Less branchy. Perhaps.

(As an aside; of course you could just test the bits one by one, but that means you always get work proportional to the number of total bits, and you still get really difficult branch prediction, so I'm not going to discuss that option.)

Now, here are a bunch of things I tried to make this work that didn't.

First, there's a way to splat the bit set into uint8_t indexes using AVX512 (after which you can iterate over them using a normal for loop); it's based on setting up a full adder-like structure and then using compressed writes. I tried it, and it was just way too slow. Geoff Langdale has the code (in a bunch of different formulations) if you'd like to look at it yourself.

So, the next natural attempt is to try to make larger blocks. If we had an uint128_t and could use that just like we did with uint64_t, we'd make life easier for the branch predictor since there would be, simply put, fewer times the inner loop would end. You can do it branch-free by means of conditional moves and such (e.g., do two bit scans, switch between them based on whether the lowest word is zero or not—similar for the other operations), and there is some support from the compiler (__uint128_t on GCC-like platforms), but in the end, going to 128 was just not enough to end up net positive.

Going to 256 or 512 wasn't easily workable; you don't have bit-scan instructions over the entire word, nor really anything like whole word subtraction. And moving data between the SIMD and integer pipes typically has a cost in itself.

So I started thinking; isn't this much of what a decompressor does? We don't really care about the higher bits of the word; as long as we can get the position of the lowest one, we don't care whether we have few or many left. So perhaps we can look at the input more like a bit stream (or byte stream) than a series of blocks; have a loop where we find the lowest bit, shift everything we just skipped or processed out, and then refill bits from the top. As always, Fabian Giesen has a thorough treatise on the subject. I wasn't concerned with squeezing every last drop out, and my data order was largely fixed anyway, so I only tried two different ways, really:

The first option is what a typical decompressor would do, except byte-based; once I've got a sufficient number of zero bits at the bottom, shift them out and reload bytes at the top. This can be done largely branch-free, so in a sense, you only have a single loop, you just keep reloading and reloading until the end. (There are at least two ways to do this reloading; you can reload only at the top, or you can reload the entire 64-bit word and mask out the bits you just read. They seemed fairly equivalent in my case.) There is a problem with the ending, though; you can read past the end. This may or may not be a problem; it was for me, but it wasn't the biggest problem (see below), so I let it be.

The other variant is somewhat more radical; I always read exactly the next 64 bits (after the previously found bit). This is done by going back to the block idea; a 64-bit word will overlap exactly one or two blocks, so we read 128 bits (two consecutive blocks) and shift the right number of bits to the right. x86 has 128-bit shifts (although they're not that fast), so this makes it fairly natural, and you can use conditional moves to make sure the second read never goes past the end of the buffer, so this feels overall like a slightly saner option.

However: None of them were faster than the normal double-loop. And I think (but never found the energy to try to positively prove) that comes down to an edge case: If there's not a single bit set in the 64-bit window, we need to handle that specially. So there we get back a fairly unpredictable branch after all—or at least, in my data set, this seems to happen fairly often. If you've got a fairly dense bit set, this won't be an issue, but then you probably have more friendly branch behavior in the loop, too. (For the reference, I have something like 3% branch misprediction overall, which is really bad when most of the stuff that I do involves ANDing bit vectors with each other!)

So, that's where I ended up. It's back to the double-loop. But perhaps someone will be able to find a magic trick that I missed. Email is welcome if you ever got this to work. :-)

25 September, 2025 09:52PM

September 24, 2025

hackergotchi for Matthew Garrett

Matthew Garrett

Investigating a forged PDF

I had to rent a house for a couple of months recently, which is long enough in California that it pushes you into proper tenant protection law. As landlords tend to do, they failed to return my security deposit within the 21 days required by law, having already failed to provide the required notification that I was entitled to an inspection before moving out. Cue some tedious argumentation with the letting agency, and eventually me threatening to take them to small claims court.

This post is not about that.

Now, under Californian law, the onus is on the landlord to hold and return the security deposit - the agency has no role in this. The only reason I was talking to them is that my lease didn't mention the name or address of the landlord (another legal violation, but the outcome is just that you get to serve the landlord via the agency). So it was a bit surprising when I received an email from the owner of the agency informing me that they did not hold the deposit and so were not liable - I already knew this.

The odd bit about this, though, is that they sent me another copy of the contract, asserting that it made it clear that the landlord held the deposit. I read it, and instead found a clause reading SECURITY: The security deposit will secure the performance of Tenant’s obligations. IER may, but will not be obligated to, apply all portions of said deposit on account of Tenant’s obligations. Any balance remaining upon termination will be returned to Tenant. Tenant will not have the right to apply the security deposit in payment of the last month’s rent. Security deposit held at IER Trust Account., where IER is International Executive Rentals, the agency in question. Why send me a contract that says you hold the money while you're telling me you don't? And then I read further down and found this:
Text reading ENTIRE AGREEMENT: The foregoing constitutes the entire agreement between the parties and may bemodified only in writing signed by all parties. This agreement and any modifications, including anyphotocopy or facsimile, may be signed in one or more counterparts, each of which will be deemed anoriginal and all of which taken together will constitute one and the same instrument. The followingexhibits, if checked, have been made a part of this Agreement before the parties’ execution:۞Exhibit 1:Lead-Based Paint Disclosure (Required by Law for Rental Property Built Prior to 1978)۞Addendum 1 The security deposit will be held by (name removed) and applied, refunded, or forfeited in accordance with the terms of this lease agreement.
Ok, fair enough, there's an addendum that says the landlord has it (I've removed the landlord's name, it's present in the original).

Except. I had no recollection of that addendum. I went back to the copy of the contract I had and discovered:
The same text as the previous picture, but addendum 1 is empty
Huh! But obviously I could just have edited that to remove it (there's no obvious reason for me to, but whatever), and then it'd be my word against theirs. However, I'd been sent the document via RightSignature, an online document signing platform, and they'd added a certification page that looked like this:
A Signature Certificate, containing a bunch of data about the document including a checksum or the original
Interestingly, the certificate page was identical in both documents, including the checksums, despite the content being different. So, how do I show which one is legitimate? You'd think given this certificate page this would be trivial, but RightSignature provides no documented mechanism whatsoever for anyone to verify any of the fields in the certificate, which is annoying but let's see what we can do anyway.

First up, let's look at the PDF metadata. pdftk has a dump_data command that dumps the metadata in the document, including the creation date and the modification date. My file had both set to identical timestamps in June, both listed in UTC, corresponding to the time I'd signed the document. The file containing the addendum? The same creation time, but a modification time of this Monday, shortly before it was sent to me. This time, the modification timestamp was in Pacific Daylight Time, the timezone currently observed in California. In addition, the data included two ID fields, ID0 and ID1. In my document both were identical, in the one with the addendum ID0 matched mine but ID1 was different.

These ID tags are intended to be some form of representation (such as a hash) of the document. ID0 is set when the document is created and should not be modified afterwards - ID1 initially identical to ID0, but changes when the document is modified. This is intended to allow tooling to identify whether two documents are modified versions of the same document. The identical ID0 indicated that the document with the addendum was originally identical to mine, and the different ID1 that it had been modified.

Well, ok, that seems like a pretty strong demonstration. I had the "I have a very particular set of skills" conversation with the agency and pointed these facts out, that they were an extremely strong indication that my copy was authentic and their one wasn't, and they responded that the document was "re-sealed" every time it was downloaded from RightSignature and that would explain the modifications. This doesn't seem plausible, but it's an argument. Let's go further.

My next move was pdfalyzer, which allows you to pull a PDF apart into its component pieces. This revealed that the documents were identical, other than page 3, the one with the addendum. This page included tags entitled "touchUp_TextEdit", evidence that the page had been modified using Acrobat. But in itself, that doesn't prove anything - obviously it had been edited at some point to insert the landlord's name, it doesn't prove whether it happened before or after the signing.

But in the process of editing, Acrobat appeared to have renamed all the font references on that page into a different format. Every other page had a consistent naming scheme for the fonts, and they matched the scheme in the page 3 I had. Again, that doesn't tell us whether the renaming happened before or after the signing. Or does it?

You see, when I completed my signing, RightSignature inserted my name into the document, and did so using a font that wasn't otherwise present in the document (Courier, in this case). That font was named identically throughout the document, except on page 3, where it was named in the same manner as every other font that Acrobat had renamed. Given the font wasn't present in the document until after I'd signed it, this is proof that the page was edited after signing.

But eh this is all very convoluted. Surely there's an easier way? Thankfully yes, although I hate it. RightSignature had sent me a link to view my signed copy of the document. When I went there it presented it to me as the original PDF with my signature overlaid on top. Hitting F12 gave me the network tab, and I could see a reference to a base.pdf. Downloading that gave me the original PDF, pre-signature. Running sha256sum on it gave me an identical hash to the "Original checksum" field. Needless to say, it did not contain the addendum.

Why do this? The only explanation I can come up with (and I am obviously guessing here, I may be incorrect!) is that International Executive Rentals realised that they'd sent me a contract which could mean that they were liable for the return of my deposit, even though they'd already given it to my landlord, and after realising this added the addendum, sent it to me, and assumed that I just wouldn't notice (or that, if I did, I wouldn't be able to prove anything). In the process they went from an extremely unlikely possibility of having civil liability for a few thousand dollars (even if they were holding the deposit it's still the landlord's legal duty to return it, as far as I can tell) to doing something that looks extremely like forgery.

There's a hilarious followup. After this happened, the agency offered to do a screenshare with me showing them logging into RightSignature and showing the signed file with the addendum, and then proceeded to do so. One minor problem - the "Send for signature" button was still there, just below a field saying "Uploaded: 09/22/25". I asked them to search for my name, and it popped up two hits - one marked draft, one marked completed. The one marked completed? Didn't contain the addendum.

comment count unavailable comments

24 September, 2025 10:22PM

hackergotchi for Philipp Kern

Philipp Kern

PSA: APT::Default-Release might be holding back updates from you

If you are like me that you are installing machines with testing and then go and flip them over to the current stable for a while using APT::Default-Release, you might not be receiving all relevant updates. In fact this setting is kind of discouraged in favor of more extensive pinning configuration.

However, the field does support regexps, so instead of just specifying, say, "trixie", you can put this in place:

APT::Default-Release "/^trixie(|-security|-proposed-updates|-updates)$/";

That should bring the security and stable updates back in.

It feels like we are recently learning a lot about the drawbacks of these overlays and how they need to be configured properly...

24 September, 2025 09:07AM by Philipp Kern (noreply@blogger.com)

September 22, 2025

hackergotchi for Evgeni Golov

Evgeni Golov

Booting Vagrant boxes with UEFI on Fedora: Permission denied

If you're still using Vagrant (I am) and try to boot a box that uses UEFI (like boxen/debian-13), a simple vagrant init boxen/debian-13 and vagrant up will entertain you with a nice traceback:

% vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Checking if box 'boxen/debian-13' version '2025.08.20.12' is up to date...
==> default: Creating image (snapshot of base box volume).
==> default: Creating domain with the following settings...
==> default:  -- Name:              tmp.JV8X48n30U_default
==> default:  -- Description:       Source: /tmp/tmp.JV8X48n30U/Vagrantfile
==> default:  -- Domain type:       kvm
==> default:  -- Cpus:              1
==> default:  -- Feature:           acpi
==> default:  -- Feature:           apic
==> default:  -- Feature:           pae
==> default:  -- Clock offset:      utc
==> default:  -- Memory:            2048M
==> default:  -- Loader:            /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/OVMF_CODE.fd
==> default:  -- Nvram:             /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/efivars.fd
==> default:  -- Base box:          boxen/debian-13
==> default:  -- Storage pool:      default
==> default:  -- Image(vda):        /home/evgeni/.local/share/libvirt/images/tmp.JV8X48n30U_default.img, virtio, 20G
==> default:  -- Disk driver opts:  cache='default'
==> default:  -- Graphics Type:     vnc
==> default:  -- Video Type:        cirrus
==> default:  -- Video VRAM:        16384
==> default:  -- Video 3D accel:    false
==> default:  -- Keymap:            en-us
==> default:  -- TPM Backend:       passthrough
==> default:  -- INPUT:             type=mouse, bus=ps2
==> default:  -- CHANNEL:             type=unix, mode=
==> default:  -- CHANNEL:             target_type=virtio, target_name=org.qemu.guest_agent.0
==> default: Creating shared folders metadata...
==> default: Starting domain.
==> default: Removing domain...
==> default: Deleting the machine folder
/usr/share/gems/gems/fog-libvirt-0.13.1/lib/fog/libvirt/requests/compute/vm_action.rb:7:in 'Libvirt::Domain#create': Call to virDomainCreate failed: internal error: process exited while connecting to monitor: 2025-09-22T10:07:55.081081Z qemu-system-x86_64: -blockdev {"driver":"file","filename":"/home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/OVMF_CODE.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/OVMF_CODE.fd': Permission denied (Libvirt::Error)
    from /usr/share/gems/gems/fog-libvirt-0.13.1/lib/fog/libvirt/requests/compute/vm_action.rb:7:in 'Fog::Libvirt::Compute::Shared#vm_action'
    from /usr/share/gems/gems/fog-libvirt-0.13.1/lib/fog/libvirt/models/compute/server.rb:81:in 'Fog::Libvirt::Compute::Server#start'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/start_domain.rb:546:in 'VagrantPlugins::ProviderLibvirt::Action::StartDomain#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/set_boot_order.rb:22:in 'VagrantPlugins::ProviderLibvirt::Action::SetBootOrder#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/share_folders.rb:22:in 'VagrantPlugins::ProviderLibvirt::Action::ShareFolders#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/prepare_nfs_settings.rb:21:in 'VagrantPlugins::ProviderLibvirt::Action::PrepareNFSSettings#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builtin/synced_folders.rb:87:in 'Vagrant::Action::Builtin::SyncedFolders#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builtin/delayed.rb:19:in 'Vagrant::Action::Builtin::Delayed#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builtin/synced_folder_cleanup.rb:28:in 'Vagrant::Action::Builtin::SyncedFolderCleanup#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/plugins/synced_folders/nfs/action_cleanup.rb:25:in 'VagrantPlugins::SyncedFolderNFS::ActionCleanup#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/prepare_nfs_valid_ids.rb:14:in 'VagrantPlugins::ProviderLibvirt::Action::PrepareNFSValidIds#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:127:in 'block in Vagrant::Action::Warden#finalize_action'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builder.rb:180:in 'Vagrant::Action::Builder#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/runner.rb:101:in 'block in Vagrant::Action::Runner#run'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/util/busy.rb:19:in 'Vagrant::Util::Busy.busy'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/runner.rb:101:in 'Vagrant::Action::Runner#run'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builtin/call.rb:53:in 'Vagrant::Action::Builtin::Call#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:127:in 'block in Vagrant::Action::Warden#finalize_action'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builder.rb:180:in 'Vagrant::Action::Builder#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/runner.rb:101:in 'block in Vagrant::Action::Runner#run'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/util/busy.rb:19:in 'Vagrant::Util::Busy.busy'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/runner.rb:101:in 'Vagrant::Action::Runner#run'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builtin/call.rb:53:in 'Vagrant::Action::Builtin::Call#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/create_network_interfaces.rb:197:in 'VagrantPlugins::ProviderLibvirt::Action::CreateNetworkInterfaces#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/create_networks.rb:40:in 'VagrantPlugins::ProviderLibvirt::Action::CreateNetworks#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/create_domain.rb:452:in 'VagrantPlugins::ProviderLibvirt::Action::CreateDomain#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/resolve_disk_settings.rb:143:in 'VagrantPlugins::ProviderLibvirt::Action::ResolveDiskSettings#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/create_domain_volume.rb:97:in 'VagrantPlugins::ProviderLibvirt::Action::CreateDomainVolume#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/handle_box_image.rb:127:in 'VagrantPlugins::ProviderLibvirt::Action::HandleBoxImage#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builtin/handle_box.rb:56:in 'Vagrant::Action::Builtin::HandleBox#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/handle_storage_pool.rb:63:in 'VagrantPlugins::ProviderLibvirt::Action::HandleStoragePool#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/set_name_of_domain.rb:34:in 'VagrantPlugins::ProviderLibvirt::Action::SetNameOfDomain#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builtin/provision.rb:80:in 'Vagrant::Action::Builtin::Provision#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.11.2/lib/vagrant-libvirt/action/cleanup_on_failure.rb:21:in 'VagrantPlugins::ProviderLibvirt::Action::CleanupOnFailure#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:127:in 'block in Vagrant::Action::Warden#finalize_action'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builder.rb:180:in 'Vagrant::Action::Builder#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/runner.rb:101:in 'block in Vagrant::Action::Runner#run'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/util/busy.rb:19:in 'Vagrant::Util::Busy.busy'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/runner.rb:101:in 'Vagrant::Action::Runner#run'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builtin/call.rb:53:in 'Vagrant::Action::Builtin::Call#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builtin/box_check_outdated.rb:93:in 'Vagrant::Action::Builtin::BoxCheckOutdated#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builtin/config_validate.rb:25:in 'Vagrant::Action::Builtin::ConfigValidate#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/warden.rb:48:in 'Vagrant::Action::Warden#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/builder.rb:180:in 'Vagrant::Action::Builder#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/runner.rb:101:in 'block in Vagrant::Action::Runner#run'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/util/busy.rb:19:in 'Vagrant::Util::Busy.busy'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/action/runner.rb:101:in 'Vagrant::Action::Runner#run'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/machine.rb:248:in 'Vagrant::Machine#action_raw'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/machine.rb:217:in 'block in Vagrant::Machine#action'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/environment.rb:631:in 'Vagrant::Environment#lock'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/machine.rb:203:in 'Method#call'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/machine.rb:203:in 'Vagrant::Machine#action'
    from /usr/share/vagrant/gems/gems/vagrant-2.3.4/lib/vagrant/batch_action.rb:86:in 'block (2 levels) in Vagrant::BatchAction#run'

The important part here is

Call to virDomainCreate failed: internal error: process exited while connecting to monitor:
2025-09-22T10:07:55.081081Z qemu-system-x86_64: -blockdev {"driver":"file","filename":"/home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/OVMF_CODE.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}:
Could not open '/home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/OVMF_CODE.fd': Permission denied (Libvirt::Error)

Of course we checked that the file permissions on this file are correct (I'll save you the ls output), so what's next? Yes, of course, SELinux!

# ausearch -m AVC
time->Mon Sep 22 12:07:55 2025
type=AVC msg=audit(1758535675.080:1613): avc:  denied  { read } for  pid=257204 comm="qemu-system-x86" name="OVMF_CODE.fd" dev="dm-2" ino=1883946 scontext=unconfined_u:unconfined_r:svirt_t:s0:c352,c717 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=0

A process in the svirt_t domain tries to access something in the user_home_t domain and is denied by the kernel. So far, SELinux is both working as designed and preventing us from doing our work, nice.

For "normal" (non-UEFI) boxes, Vagrant uploads the image to libvirt, which stores it in ~/.local/share/libvirt/images/ and boots fine from there. For UEFI boxen, one also needs loader and nvram files, which Vagrant keeps in ~/.vagrant.d/boxes/<box_name> and that's what explodes in our face here.

As ~/.local/share/libvirt/images/ works well, and is labeled svirt_home_t let's see what other folders use that label:

# semanage fcontext -l |grep svirt_home_t
/home/[^/]+/\.cache/libvirt/qemu(/.*)?             all files          unconfined_u:object_r:svirt_home_t:s0
/home/[^/]+/\.config/libvirt/qemu(/.*)?            all files          unconfined_u:object_r:svirt_home_t:s0
/home/[^/]+/\.libvirt/qemu(/.*)?                   all files          unconfined_u:object_r:svirt_home_t:s0
/home/[^/]+/\.local/share/gnome-boxes/images(/.*)? all files          unconfined_u:object_r:svirt_home_t:s0
/home/[^/]+/\.local/share/libvirt/boot(/.*)?       all files          unconfined_u:object_r:svirt_home_t:s0
/home/[^/]+/\.local/share/libvirt/images(/.*)?     all files          unconfined_u:object_r:svirt_home_t:s0

Okay, that all makes sense, and it's just missing the Vagrant-specific folders!

# semanage fcontext -a -t svirt_home_t '/home/[^/]+/\.vagrant.d/boxes(/.*)?'

Now relabel the Vagrant boxes:

% restorecon -rv ~/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13
Relabeled /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13 from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:svirt_home_t:s0
Relabeled /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/metadata_url from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:svirt_home_t:s0
Relabeled /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12 from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:svirt_home_t:s0
Relabeled /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:svirt_home_t:s0
Relabeled /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/box_0.img from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:svirt_home_t:s0
Relabeled /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/metadata.json from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:svirt_home_t:s0
Relabeled /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/Vagrantfile from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:svirt_home_t:s0
Relabeled /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/OVMF_CODE.fd from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:svirt_home_t:s0
Relabeled /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/OVMF_VARS.fd from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:svirt_home_t:s0
Relabeled /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/box_update_check from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:svirt_home_t:s0
Relabeled /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/efivars.fd from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:svirt_home_t:s0

And it works!

% vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Checking if box 'boxen/debian-13' version '2025.08.20.12' is up to date...
==> default: Creating image (snapshot of base box volume).
==> default: Creating domain with the following settings...
==> default:  -- Name:              tmp.JV8X48n30U_default
==> default:  -- Description:       Source: /tmp/tmp.JV8X48n30U/Vagrantfile
==> default:  -- Domain type:       kvm
==> default:  -- Cpus:              1
==> default:  -- Feature:           acpi
==> default:  -- Feature:           apic
==> default:  -- Feature:           pae
==> default:  -- Clock offset:      utc
==> default:  -- Memory:            2048M
==> default:  -- Loader:            /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/OVMF_CODE.fd
==> default:  -- Nvram:             /home/evgeni/.vagrant.d/boxes/boxen-VAGRANTSLASH-debian-13/2025.08.20.12/libvirt/efivars.fd
==> default:  -- Base box:          boxen/debian-13
==> default:  -- Storage pool:      default
==> default:  -- Image(vda):        /home/evgeni/.local/share/libvirt/images/tmp.JV8X48n30U_default.img, virtio, 20G
==> default:  -- Disk driver opts:  cache='default'
==> default:  -- Graphics Type:     vnc
==> default:  -- Video Type:        cirrus
==> default:  -- Video VRAM:        16384
==> default:  -- Video 3D accel:    false
==> default:  -- Keymap:            en-us
==> default:  -- TPM Backend:       passthrough
==> default:  -- INPUT:             type=mouse, bus=ps2
==> default:  -- CHANNEL:             type=unix, mode=
==> default:  -- CHANNEL:             target_type=virtio, target_name=org.qemu.guest_agent.0
==> default: Creating shared folders metadata...
==> default: Starting domain.
==> default: Domain launching with graphics connection settings...
==> default:  -- Graphics Port:      5900
==> default:  -- Graphics IP:        127.0.0.1
==> default:  -- Graphics Password:  Not defined
==> default:  -- Graphics Websocket: 5700
==> default: Waiting for domain to get an IP address...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 192.168.124.157:22
    default: SSH username: vagrant
    default: SSH auth method: private key
    default:
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default:
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!

22 September, 2025 10:37AM by evgeni

Russell Coker

More About the Colmi P80

The FOSS Android program for communicating with smart watches is Gadget Bridge which now has support for the Colmi P80 [1].

I first blogged about the Colmi P80 just over a month ago [2]. Now I have a couple of relatives using it happily on Android with the proprietary app. I couldn’t use it myself because I require more control over which apps have their notifications go to the watch than the Colmi app offers. Also I’m trying to move away from non-free software.

Yesterday the f-droid repository informed me that there was a new version of Gadget Bridge and the changelog indicated support for the Colmi P80 so I connected the P80 and disconnected the PineTime.

The first problem I noticed is that the vibrator on the P80 when on it’s maximum setting is much weaker than that on the PineTime, so weak that I often didn’t notice it. Maybe if I wore it for a few weeks I would teach myself to notice it but it should just be able to work with me on this. If it could be set to have multiple bursts of vibrating then that would work.

The next problem is that the P80 by default does not turn the screen on when there’s a notification and there seems to be no way to configure it to do so. I configured it to turn on when I raise my arm which can mostly work but that still relies on me noticing the vibration. Vibration and the screen light turning on would be harder to miss than vibration on it’s own.

I don’t recall seeing any review of smart watches ever that stated whether the screen would turn on when there’s a notification or whether the vibration was easy to notice.

One problem with both the PineTime (running InfiniTime) and the P80 is that when the screen is turned on (through gesture, pushing the button, or a notification in the case of the Pinetime) it is active for swiping to change the settings. I would like to have some other action required before settings can be changed so that if the screen turns on when I’m asleep my watch won’t brush against something and change it’s settings (which has happened).

It’s neat how Gadget Bridge supports talking to multiple smart watches at the same time. One useful feature for that would be to have different notification settings for each watch. I can imagine someone changing between a watch for jogging and a watch for work and wanting different settings.

Colmi P80 Problems

No authentication for Bluetooth connections.

Runs non-free software so no chance to fix things.

Battery life worse than PineTime (but not really bad).

Vibration weak.

Screen doesn’t turn on when notification is sent.

Conclusion

I’m using the PineTime as my daily driver again. While it works well enough for some people (even with the Colmi proprietary app) it doesn’t do what I want. It is however a good test device for FOSS work on the phone side, it has a decent feature set and is cheap.

Apart from lack of authentication and running non-free software the problems are mostly a matter of taste. Some people might think it’s great the way it works.

22 September, 2025 08:39AM by etbe

Vincent Bernat

Akvorado release 2.0

Akvorado 2.0 was released today! Akvorado collects network flows with IPFIX and sFlow. It enriches flows and stores them in a ClickHouse database. Users can browse the data through a web console. This release introduces an important architectural change and other smaller improvements. Let’s dive in! 🤿

$ git diff --shortstat v1.11.5
 493 files changed, 25015 insertions(+), 21135 deletions(-)

New “outlet� service

The major change in Akvorado 2.0 is splitting the inlet service into two parts: the inlet and the outlet. Previously, the inlet handled all flow processing: receiving, decoding, and enrichment. Flows were then sent to Kafka for storage in ClickHouse:

Akvorado flow processing before the change: flows are received and processed by the inlet, sent to Kafka and stored in ClickHouse
Akvorado flow processing before the introduction of the outlet service

Network flows reach the inlet service using UDP, an unreliable protocol. The inlet must process them fast enough to avoid losing packets. To handle a high number of flows, the inlet spawns several sets of workers to receive flows, fetch metadata, and assemble enriched flows for Kafka. Many configuration options existed for scaling, which increased complexity for users. The code needed to avoid blocking at any cost, making the processing pipeline complex and sometimes unreliable, particularly the BMP receiver.1 Adding new features became difficult without making the problem worse.2

In Akvorado 2.0, the inlet receives flows and pushes them to Kafka without decoding them. The new outlet service handles the remaining tasks:

Akvorado flow processing after the change: flows are received by the inlet, sent to Kafka, processed by the outlet and inserted in ClickHouse
Akvorado flow processing after the introduction of the outlet service

This change goes beyond a simple split:3 the outlet now reads flows from Kafka and pushes them to ClickHouse, two tasks that Akvorado did not handle before. Flows are heavily batched to increase efficiency and reduce the load on ClickHouse using ch-go, a low-level Go client for ClickHouse. When batches are too small, asynchronous inserts are used (e20645). The number of outlet workers scales dynamically (e5a625) based on the target batch size and latency (50,000 flows and 5 seconds by default).

This new architecture also allows us to simplify and optimize the code. The outlet fetches metadata synchronously (e20645). The BMP component becomes simpler by removing cooperative multitasking (3b9486). Reusing the same RawFlow object to decode protobuf-encoded flows from Kafka reduces pressure on the garbage collector (8b580f).

The effect on Akvorado’s overall performance was somewhat uncertain, but a user reported 35% lower CPU usage after migrating from the previous version, plus resolution of the long-standing BMP component issue. 🥳

Other changes

This new version includes many miscellaneous changes, such as completion for source and destination ports (f92d2e), and automatic restart of the orchestrator service (0f72ff) when configuration changes to avoid a common pitfall for newcomers.

Let’s focus on some key areas for this release: observability, documentation, CI, Docker, Go, and JavaScript.

Observability

Akvorado exposes metrics to provide visibility into the processing pipeline and help troubleshoot issues. These are available through Prometheus HTTP metrics endpoints, such as /api/v0/inlet/metrics. With the introduction of the outlet, many metrics moved. Some were also renamed (4c0b15) to match Prometheus best practices. Kafka consumer lag was added as a new metric (e3a778).

If you do not have your own observability stack, the Docker Compose setup shipped with Akvorado provides one. You can enable it by activating the profiles introduced for this purpose (529a8f).

The prometheus profile ships Prometheus to store metrics and Alloy to collect them (2b3c46, f81299, and 8eb7cd). Redis and Kafka metrics are collected through the exporter bundled with Alloy (560113). Other metrics are exposed using Prometheus metrics endpoints and are automatically fetched by Alloy with the help of some Docker labels, similar to what is done to configure Traefik. cAdvisor was also added (83d855) to provide some container-related metrics.

The loki profile ships Loki to store logs (45c684). While Alloy can collect and ship logs to Loki, its parsing abilities are limited: I could not find a way to preserve all metadata associated with structured logs produced by many applications, including Akvorado. Vector replaces Alloy (95e201) and features a domain-specific language, VRL, to transform logs. Annoyingly, Vector currently cannot retrieve Docker logs from before it was started.

Finally, the grafana profile ships Grafana, but the shipped dashboards are broken. This is planned for a future version.

Documentation

The Docker Compose setup provided by Akvorado makes it easy to get the web interface up and running quickly. However, Akvorado requires a few mandatory steps to be functional. It ships with comprehensive documentation, including a chapter about troubleshooting problems. I hoped this documentation would reduce the support burden. It is difficult to know if it works. Happy users rarely report their success, while some users open discussions asking for help without reading much of the documentation.

In this release, the documentation was significantly improved.

$ git diff --shortstat v1.11.5 -- console/data/docs
 10 files changed, 1873 insertions(+), 1203 deletions(-)

The documentation was updated (fc1028) to match Akvorado’s new architecture. The troubleshooting section was rewritten (17a272). Instructions on how to improve ClickHouse performance when upgrading from versions earlier than 1.10.0 was added (5f1e9a). An LLM proofread the entire content (06e3f3). Developer-focused documentation was also improved (548bbb, e41bae, and 871fc5).

From a usability perspective, table of content sections are now collapsable (c142e5). Admonitions help draw user attention to important points (8ac894).

Admonition in Akvorado documentation to ask a user not to open an issue or start a discussion before reading the documentation
Example of use of admonitions in Akvorado's documentation

Continuous integration

This release includes efforts to speed up continuous integration on GitHub. Coverage and race tests run in parallel (6af216 and fa9e48). The Docker image builds during the tests but gets tagged only after they succeed (8b0dce).

GitHub workflow for CI with many jobs, some of them running in parallel, some not
GitHub workflow to test and build Akvorado

End-to-end tests (883e19) ensure the shipped Docker Compose setup works as expected. Hurl runs tests on various HTTP endpoints, particularly to verify metrics (42679b and 169fa9). For example:

## Test inlet has received NetFlow flows
GET http://127.0.0.1:8080/prometheus/api/v1/query
[Query]
query: sum(akvorado_inlet_flow_input_udp_packets_total{job="akvorado-inlet",listener=":2055"})
HTTP 200
[Captures]
inlet_receivedflows: jsonpath "$.data.result[0].value[1]" toInt
[Asserts]
variable "inlet_receivedflows" > 10

## Test inlet has sent them to Kafka
GET http://127.0.0.1:8080/prometheus/api/v1/query
[Query]
query: sum(akvorado_inlet_kafka_sent_messages_total{job="akvorado-inlet"})
HTTP 200
[Captures]
inlet_sentflows: jsonpath "$.data.result[0].value[1]" toInt
[Asserts]
variable "inlet_sentflows" >= {{ inlet_receivedflows }}

Docker

Akvorado ships with a comprehensive Docker Compose setup to help users get started quickly. It ensures a consistent deployment, eliminating many configuration-related issues. It also serves as a living documentation of the complete architecture.

This release brings some small enhancements around Docker:

Previously, many Docker images were pulled from the Bitnami Containers library. However, VMWare acquired Bitnami in 2019 and Broadcom acquired VMWare in 2023. As a result, Bitnami images were deprecated in less than a month. This was not really a surprise4. Previous versions of Akvorado had already started moving away from them. In this release, the Apache project’s Kafka image replaces the Bitnami one (1eb382). Thanks to the switch to KRaft mode, Zookeeper is no longer needed (0a2ea1, 8a49ca, and f65d20).

Akvorado’s Docker images were previously compiled with Nix. However, building AArch64 images on x86-64 is slow because it relies on QEMU userland emulation. The updated Dockerfile uses multi-stage and multi-platform builds: one stage builds the JavaScript part on the host platform, one stage builds the Go part cross-compiled on the host platform, and the final stage assembles the image on top of a slim distroless image (268e95 and d526ca).

# This is a simplified version
FROM --platform=$BUILDPLATFORM node:20-alpine AS build-js
RUN apk add --no-cache make
WORKDIR /build
COPY console/frontend console/frontend
COPY Makefile .
RUN make console/data/frontend

FROM --platform=$BUILDPLATFORM golang:alpine AS build-go
RUN apk add --no-cache make curl zip
WORKDIR /build
COPY . .
COPY --from=build-js /build/console/data/frontend console/data/frontend
RUN go mod download
RUN make all-indep
ARG TARGETOS TARGETARCH TARGETVARIANT VERSION
RUN make

FROM gcr.io/distroless/static:latest
COPY --from=build-go /build/bin/akvorado /usr/local/bin/akvorado
ENTRYPOINT [ "/usr/local/bin/akvorado" ]

When building for multiple platforms with --platform linux/amd64,linux/arm64,linux/arm/v7, the build steps until the highlighted line execute only once for all platforms. This significantly speeds up the build. 🚅

Akvorado now ships Docker images for these platforms: linux/amd64, linux/amd64/v3, linux/arm64, and linux/arm/v7. When requesting ghcr.io/akvorado/akvorado, Docker selects the best image for the current CPU. On x86-64, there are two choices. If your CPU is recent enough, Docker downloads linux/amd64/v3. This version contains additional optimizations and should run faster than the linux/amd64 version. It would be interesting to ship an image for linux/arm64/v8.2, but Docker does not support the same mechanism for AArch64 yet (792808).

Go

This release includes many changes related to Go but not visible to the users.

Toolchain

In the past, Akvorado supported the two latest Go versions, preventing immediate use of the latest enhancements. The goal was to allow users of stable distributions to use Go versions shipped with their distribution to compile Akvorado. However, this became frustrating when interesting features, like go tool, were released. Akvorado 2.0 requires Go 1.25 (77306d) but can be compiled with older toolchains by automatically downloading a newer one (94fb1c).5 Users can still override GOTOOLCHAIN to revert this decision. The recommended toolchain updates weekly through CI to ensure we get the latest minor release (5b11ec). This change also simplifies updates to newer versions: only go.mod needs updating.

Thanks to this change, Akvorado now uses wg.Go() (77306d) and I have started converting some unit tests to the new test/synctest package (bd787e, 7016d8, and 159085).

Testing

When testing equality, I use a helper function Diff() to display the differences when it fails:

got := input.Keys()
expected := []int{1, 2, 3}
if diff := helpers.Diff(got, expected); diff != "" {
    t.Fatalf("Keys() (-got, +want):\n%s", diff)
}

This function uses kylelemons/godebug. This package is no longer maintained and has some shortcomings: for example, by default, it does not compare struct private fields, which may cause unexpectedly successful tests. I replaced it with google/go-cmp, which is stricter and has better output (e2f1df).

Another package for Kafka

Another change is the switch from Sarama to franz-go to interact with Kafka (756e4a and 2d26c5). The main motivation for this change is to get a better concurrency model. Sarama heavily relies on channels and it is difficult to understand the lifecycle of an object handed to this package. franz-go uses a more modern approach with callbacks6 that is both more performant and easier to understand. It also ships with a package to spawn fake Kafka broker clusters, which is more convenient than the mocking functions provided by Sarama.

Improved routing table for BMP

To store its routing table, the BMP component used kentik/patricia, an implementation of a patricia tree focused on reducing garbage collection pressure. gaissmai/bart is a more recent alternative using an adaptation of [Donald Knuth’s ART algorithm][] that promises better performance and delivers it: 90% faster lookups and 27% faster insertions (92ee2e and fdb65c).

Unlike kentik/patricia, gaissmai/bart does not help efficiently store values attached to each prefix. I adapted the same approach as kentik/patricia to store route lists for each prefix: store a 32-bit index for each prefix, and use it to build a 64-bit index for looking up routes in a map. This leverages Go’s efficient map structure.

gaissmai/bart also supports a lockless routing table version, but this is not simple because we would need to extend this to the map storing the routes and to the interning mechanism. I also attempted to use Go’s new unique package to replace the intern package included in Akvorado, but performance was worse.7

Miscellaneous

Previous versions of Akvorado were using a custom Protobuf encoder for performance and flexibility. With the introduction of the outlet service, Akvorado only needs a simple static schema, so this code was removed. However, it is possible to enhance performance with planetscale/vtprotobuf (e49a74, and 8b580f). Moreover, the dependency on protoc, a C++ program, was somewhat annoying. Therefore, Akvorado now uses buf, written in Go, to convert a Protobuf schema into Go code (f4c879).

Another small optimization to reduce the size of the Akvorado binary by 10 MB was to compress the static assets embedded in Akvorado in a ZIP file. It includes the ASN database, as well as the SVG images for the documentation. A small layer of code makes this change transparent (b1d638 and e69b91).

JavaScript

Recently, two large supply-chain attacks hit the JavaScript ecosystem: one affecting the popular packages chalk and debug and another impacting the popular package @ctrl/tinycolor. These attacks also exist in other ecosystems, but JavaScript is a prime target due to heavy use of small third-party dependencies. The previous version of Akvorado relied on 653 dependencies.

npm-run-all was removed (3424e8, 132 dependencies). patch-package was removed (625805 and e85ff0, 69 dependencies) by moving missing TypeScript definitions to env.d.ts. eslint was replaced with oxlint, a linter written in Rust (97fd8c, 125 dependencies, including the plugins).

I switched from npm to Pnpm, an alternative package manager (fce383). Pnpm does not run install scripts by default8 and prevents installing packages that are too recent. It is also significantly faster.9 Node.js does not ship Pnpm but it ships Corepack, which allows us to use Pnpm without installing it. Pnpm can also list licenses used by each dependency, removing the need for license-compliance (a35ca8, 42 dependencies).

For additional speed improvements, beyond switching to Pnpm and Oxlint, Vite was replaced with its faster Rolldown version (463827).

After these changes, Akvorado “only� pulls 225 dependencies. 😱

Next steps

I would like to land three features in the next version of Akvorado:

  • Add Grafana dashboards to complete the observability stack. See issue #1906 for details.

  • Integrate OVH’s Grafana plugin by providing a stable API for such integrations. Akvorado’s web console would still be useful for browsing results, but if you want to build and share dashboards, you should switch to Grafana. See issue #1895.

  • Move some work currently done in ClickHouse (custom dictionaries, GeoIP and IP enrichment) back into the outlet service. This should give more flexibility for adding features like the one requested in issue #1030. See issue #2006.


I started working on splitting the inlet into two parts more than one year ago. I found more motivation in recent months, partly thanks to Claude Code, which I used as a rubber duck. Almost none of the produced code was kept:10 it is like an intern who does not learn. 🦆


  1. Many attempts were made to make the BMP component both performant and not blocking. See for example PR #254, PR #255, and PR #278. Despite these efforts, this component remained problematic for most users. See issue #1461 as an example. ↩�

  2. Some features have been pushed to ClickHouse to avoid the processing cost in the inlet. See for example PR #1059. ↩�

  3. This is the biggest commit:

    $ git show --shortstat ac68c5970e2c | tail -1
    231 files changed, 6474 insertions(+), 3877 deletions(-)
    

    ↩�

  4. Broadcom is known for its user-hostile moves. Look at what happened with VMWare. ↩�

  5. As a Debian developer, I dislike these mechanisms that circumvent the distribution package manager. The final straw came when Go 1.25 spent one month in the Debian NEW queue, an arbitrary mechanism I don’t like at all. ↩�

  6. In the early years of Go, channels were heavily promoted. Sarama was designed during this period. A few years later, a more nuanced approach emerged. See notably “Go channels are bad and you should feel bad.� ↩�

  7. This should be investigated further, but my theory is that the intern package uses 32-bit integers, while unique uses 64-bit pointers. See commit 74e5ac. ↩�

  8. This is also possible with npm. See commit dab2f7. ↩�

  9. An even faster alternative is Bun, but it is less available. ↩�

  10. The exceptions are part of the code for the admonition blocks, the code for collapsing the table of content, and part of the documentation. ↩�

22 September, 2025 08:12AM by Vincent Bernat