Friday, June 9, 2023

The 20th century tank is obsolete on the battlefield

This blog post is a little different from my usual content, I know, but it relates to technology, and specifically, military technology. As we have seen in the Russo-Ukrainian war, the 20th century tank is increasingly obsolete on the battlefield. In the past, the biggest threats to tanks were artillery, mines, and other tanks. This has not changed much, but the modern battlefield introduces two new threats: man-portable anti-tank guided missiles like the NLAW and the Javelin; and drones of various types, be it kamikaze, long-rate high-altitude drones like the Bayraktar, or quadcopters with grenades.

As a result, tanks have become hopelessly vulnerable, facing threats from the air, the ground, infantry and long-rate artillery. But believe it or not, this situation is not without historical precedent. When the Leopard 1 and AMX-30 were developed, the threat landscape was similarly dire; instead of heavier armour, designers focused on lighter vehicles that were more mobile, and which could gain an advantage in firepower. This advantage didn’t necessarily come in the form of high-calibre guns (the 105mm L7 gun was more than sufficient) but in the form of night vision, stabilisation, and ballistic computers.

If you are expecting this piece to argue that tanks should ditch armour and focus on being light—or that MBTs should be swapped for armoured recoinaissance vehicles like the AMX—then you’d be wrong. Tanks need to be redesigned for the 21st century, and while, yes, they should be lighter (perhaps in the 45 tonne range rather than 70 tonnes) the real solution to the problem is much more sophisticated.

Active Protection Systems

APS systems like the Rafael Trophy have proven very successful, and armies have recognised this potential, with Germany and the US retroffitting their tanks with these systems. APS systems are effective against the biggest killer of tanks in Ukraine: man-portable ATGMs. However, they are not enough on their own, and particularly, they still leave the tank vulnerable to top-attacks, i.e. drones.

How to counter drones?

When we say drones, we are referring to a large variety of systems. It is not realistic, for example, to expect a tank to defend against Reaper or Predator drones, as these will fire long-range missiles from high altitudes (that is the job of air-defence systems like IRIS-T or Patriot). However, we do need to protect tanks from ordinary sons-of-bitches like quadcopters, which might only cost $80,000 rather than millions. These systems fly lower, and have to be directly above their target to drop their payload.

Perhaps the simplest solution is tactical: have a SPAAG like the Gepard accompany each tank squad. The flak gun defends the tanks and APCs from aerial attack, while the other armoured vehicles focus on ground threats. In my opinion, this is the best solution in the short-term.

Another solution is to outfit a tank with its own air-defence system and radar. This had advantages and disadvantages. The advantage is that it makes the tank more independent, rather than relying on a SPAAG which could be shot by other tanks, or the drone itself. The disadvantage is primarily the increased cost and complexity of accommodating such a system. Also, the radar would give the tank’s position away.

The air defence system could be in the form of missiles like Starstreaker or Mistral, or it could be as simple as repurposing the tank’s machine gun. (Some tanks have two machine guns.) A machine gun firing high-velocity 5.56 or 7.62 ammunition (in the 1000 m/s range) would be capable of taking down drones at altitudes up to 500m or so. While a missile is more capable, it costs far more; it doesn’t make sense to expend a $150,000 missile to take out a drone costing as little as $40,000. Or, it might do, if it protects a multi-million dollar tank.

Stealth

The Challenger 2 is one of the few tanks to have been designed with a low radar signature. It’s a step in the right direction, but it’s not enough. Tanks should be designed with stealth in mind, as it will help them evade drones and artillery. Camouflage is one obvious way to do this; tanks hidden in forests can be very hard to spot. But the biggest problem is probably the IR signature (and noise) of the diesel engine.

I think electric tanks are a non-starter because the infrastructure doesn’t exist to support them, and range will be a problem. However, hybrid diesel-electric tanks could work very well indeed. They can use the diesel engine when on the move over long distances, and rely on electric power for stealth and urban combat.

Armour considerations

I think a tank, at least in its base configuration, should not expect survive anything bigger than a 30mm auto-cannon. From the front, it may be possible to armour it against 105mm and maybe 120mm calibre ammunition, because of the slope; but it any case not from the sides and rear.

Instead, tanks should think more about their top armour. In Ukraine, we have seen a PzH 2000 (an armoured howitzer) survive a top-attack from a kamikaze drone with a payload in the 4kg range. The howitzer was damaged, but the damage was repairable and all the crew survived. This is very impressive, and it was made possible thanks to “hedgehog” armour: rubber spikes on the top. This should become a standard applique on MBTs.

Another design consideration is that ammunition should be stored separately from the turret, as ammunition cooking off makes a hit from an ATGM or similar munition more deadly. Better to lose one man than the whole crew.

Thursday, June 8, 2023

Think twice before working at a startup

I’m a machine learning professional who has worked for a startup in the past. I’m writing from an EU and UK perspective, but generally speaking, tech startups are very American in the way they operate. This can result in some nasty shocks. This post will give you an honest take on the upsides of working at a startup—and the downsides.

Myth vs reality

Years of media coverage, often written by journalists who have never worked in a startup, have painted a very rosy picture, which I fell for in the early years of my career. The image of the startup is that of a small, Agile company that isn’t afraid to innovate, unlike big bureaucratic corporations. The startup is typically located in a sexy area (like a garage in Silicon Valley); likewise, the work they do is sexy. Your colleagues will become your friends. You can play videogames or ping pong, and drink beers. The startup is diverse, with people of many different nationalities and backgrounds.

And all of this is true, but this is a classic example of lying by omission. Here are the realities:

  1. Your colleagues may be from diverse nationalities and/or ethnic origin, but they will almost all be men between the age of 25–40. They are also less likely to be parents. If you’re young, you’re considered too inexperienced to work there, and if you’re old, forget about it.
  2. You may drink beers and play video games, but on a Friday afternoon after 5pm, when you should be enjoying your weekend. The other 4 days of the week will be spent frantically working, often with longer hours than well-established players. Early stage startups, and senior employees, might spend time working on weekends and holidays.
  3. Sexy areas = high cost of living, and startups are often resistant to remote work policies (despite being “innovative”).

It doesn’t get better, I’m afraid.

Startups and the Agile methodology

Startups like to bill themselves as Agile, which means (roughly translated for non-software folks out there) as doing things interatively and not getting trapped in analysis paralysis. What actually happens is that startups write a lot of untested code which constantly breaks. The requirements change all the time, and not always because of good reasons (like the client wanting something else) but because management is fickle. There is often not much time for a formal design process, which results in poor abstractions; and nobody is responsible for writing documentation or doing QA. Likewise, there was no model review at the startup I worked for.

If you want to learn industry best practices, you are usually better off elsewhere. I would be especially careful about any startup that operates in healthcare, defence or finance. “Move fast and break things” might work for a social media site, but it will be a disaster in these fields.

Are startups innovative?

Some startups genuinely do innovative work that pushes the envelope of software engineering and/or machine learning. But this is the exception rather than the rule. Plenty of startups are building just another travel app, ecommerce site or payments platform – and it’s often a product that nobody wants (which is the reason so many startups fail). In machine learning, another extreme is the startup that tries pie-in-the-sky ideas which are academic projects, not serious commercial ideas.

And there is plenty of anodyne work in startups. You’ll end up doing the same thing as in a big company, but with worse management, benefits, and (especially) job security.

Furthermore, tech startups can often have messianic leaders. I personally find this very annoying. Startups are rarely solving world hunger or climate change, and no, your smartphone app doesn’t count. This particular criticism can be levelled at tech workers in general, not just startups, but I find many techies boring. They talk about tech during dinner parties or drinks, and rarely know much about current affairs or art. At best, I can talk about my gym routine with them.

Management can suck

Of course, management can suck in any company. But startups are often run by people who have no management experience. Furthermore, startups have a tendency to be extremely disorganised.

Job security sucks

Job security sucks for two reasons. One is that startups can go bankrupt quite quickly – they have a high burn rate, and are very dependent on investor capital, often with few clients and not enough revenue to be profitable. The second reason is that startups can be very erratic and unpredictable – and they operate with a short-term mindset. They might not need your skillset in six months or even weeks (!) from now. They don’t have the long-term mindset of keeping employees around so that they can train them.

This mindset results in a litany of problems:

  1. Lots of technical debt, with complex systems in production and not many people that know how they work.
  2. High turnover is demoralising, and a lot of people decide to leave fearing they will be next.
  3. An inability to gain a competitive advantage by building up in-house know-how.

(If you are wondering, yes, I have seen a startup fire an entire department weeks after hiring them.)

From the perspective of the employee, it will be stressful, especially if you have children depending on you to put food on the table.

Is there any reason you should work for a startup?

There are some. Let’s get the bad reason out of the way: while you may get rich, it’s unlikely. What you do get out of working in a startup is lots of experience with a wide variety of technologies, which you won’t get if you’re pigeonholed somewhere in a big company. You will learn faster. And you will often get a stronger grounding in business problems (if you’re a technical worker) or technology (if you’re from a business background). Finally, there will be less red-tape at a startups; you will have greater freedom to try different things.

The compensation varies but I have not found startup salaries to be markedly different from the industry norm. You might want to work for a startup if they give you the best compensation (but keep in mind the downsides).

Startups from an EU perspective

Pundits admonish the EU for having way fewer tech startups than the US. Frankly, I’m not sure that’s such a bad thing. Startups:

  1. Can burn millions, yet never become profitable even years after being founded. The startup I worked for wasn’t profitable in 3 years after being founded, and didn’t plan to be profitable until 5 years in the future.
  2. Can cost the clients money when their product fails unexpectedly.
  3. Most never develop a useful product.

I am leaning more towards the Nordic/German/Benelux model of SMEs. It’s better to start small, serving a niche, and then expand, than to attempt something grandiose.

Wednesday, October 5, 2022

Would the real data scientists please stand up?

There are a number of myths surrounding the field of data science, the jobs available, and the pay. When I was in my salad years and green with judgement (read: before I started my master’s) I was lead to believe that data scientists are brilliant individuals who go on to earn $10,000 a month right out of school—provided that they can pass the rigorous demands of their educational programme.

The reality is much more nuanced. It is true that there are a small number of graduates who earn that much straight out of school, along with a somewhat larger number of experienced individuals (e.g. Head of Data, Senior Machine Learning Engineer, Data Science Lead). However, there are many things they don’t tell you:

  1. Those kinds of salaries are rarely found outside of the US, and since most data scientists have at least a master’s degree, that often means debt—as much as $100,000, for which the Federal interest rate is a criminal 6%;
  2. They typically go only to graduates of the top schools;
  3. They are typically only for on-site roles in some very expensive cities, e.g. San Francisco, New York and maybe London, where rents are sky-high;
  4. They are mostly offered by brand-name tech companies, finance firms, and a few startups; the recruitment process tends to be arduous. It often involves a battery of tests in coding, maths, IQ, and personality tests, in addition to multiple interviews. And by “interview”, think live-coding.
  5. By “right out of school”, think PhD. A master’s is just a pre-requisite in many firms.

So clearly, data scientists being paid $100,000+ per year are the minority. If you look at job sites like Glassdoor, you’d think that this is the median salary, but these sites suffer from sampling bias. The actual median salary in the UK is little over 50K GBP. Comparing salaries across countries is difficult because:

  1. Exchange rates fluctuate, and purchasing power parity needs to be taken into account;
  2. In the US, employers pay 7.65% social security taxes; in European countries, it’s usually higher, e.g. 15% in the UK, 25% Belgium.
  3. Rents vary.
  4. Student loans vary dramatically.

So what are some typical salary bands, responsibilities, and what kind of skills are actually required? If you think that you can get a six figure job just by knowing how to work with sklearn, pandas and matplotlib, you are sadly mistaken. These are simply the basic requirements for an entry-level job. (And not even all the basics, at that.)

In order from highest to lowest:

  • $100,000+ per year, or the PPP equivalent in euros and pounds: There are 3 types of people, in my experience.
    • The most “classical” profile is someone with a PhD in machine learning/AI from a top school who has written a few white papers in novel ML algorithms. These people work in really cutting-edge applications.
    • The second kind of person is someone with at least a couple of years of experience who is very proficient at programming, and competent with many technologies: not just Python, but they may also be crack C++, Java or Julia programmers. They know their way around cloud computing, containerisation, distributed computing, and have read a book or two on design patterns. They know SQL and NoSQL. These people productionize ML models. They are basically glorified software engineers.
    • Experienced personnel with deep domain knowledge in medicine, business, finance, or linguistics who also have the technical skills.
  • Anywhere from $60K to less than $100K: These are either PhD grads from reputable-but-less-than-top universities (those other than MIT, Oxbridge, Ivy League), or ML engineers with some experience. Fresh PhD grads will start on the low end of the scale.
  • From $40K to $60K or PPP equivalent: They know the basics, I guess. Strong Python/R skills and stats. Decent SQL skills. Can wrangle their data, write OOP code, and knows how to setup a Docker container or write a shell script. Fresh grads start on the low end, but salaries increase quickly with a bit of experience.
  • Less than $40K: These are typically called data analysts. They might write a Python script or two.
  • Unemployed (or not employed as a data scientist): The legions of wannabes from bootcamps, online certificates, and 1 year degrees who write awful code (usually in Jupyter notebooks), don’t understand statistics, don’t know how to query a database, clean data, or understand how to properly evaluate the output of a model.

Some other things to discuss. I hear a lot of people mention domain knowledge; often, these people have a lot of domain knowledge in their field of expertise and are trying to break into data science. I hate to break it to them, but while domain knowledge is valuable, all the domain knowledge in the world won’t help if you don’t the master the basics of programming and statistical analysis. It’s usually the coding part where they struggle.

Is there a shortage of data scientists in the industry? Ten years ago, definitely. Reputable sources still claim that anywhere from 20% to 50% of companies struggle to fill data science roles, but take these numbers with a pinch of salt.

What I can say is that there’s definitely a shortage of talented, experienced data scientists who are willing to work for less than six figures. (The competition for six figure roles is blisteringly intense, with many talented, qualified individuals applying for each role.) The shortage is particularly acute for “full-stack” data scientists who can productionize models and do data engineering. However, there seems to be a big oversupply of bootcamp/online course graduates, as well as degree holders from orthogonal disciplines. The life and social sciences are the worst offenders. A bit of data analysis in SPSS, or rookie Python ability probability won’t net you a job, or a low-paying one at best.

If you are looking to enter the field like I am, my advice is to start by getting a degree in DS, CS or statistics (but make sure it’s a stats course with strong programming requirements). It’s generally easier to become a data scientist by first starting out as a developer or a data engineer (glorified developer). By the way, a position as a data analyst does not prepare you for the reality of data science or software engineering. The stories you hear about data analysts becoming data scientists was from years ago when management didn’t know the difference – or they have the title data scientist but only do analytics work. A data analyst is usually a glorified Excel monkey who maybe knows some Python and SQL. While they might be able to create a simple ML model, they often don't really understand how to interpret the model or tune it, as their statistics knowledge only covers the basics. The learning curve for software engineering is even steeper and such a person can only do their job if they have an army of data engineers and ML engineers working for them.

Although companies like IBM tell us that data is a fast-growing field, the threat of automation should not be discounted. In 10 years, there might not be a market for data scientists, or at least, it won’t employ large numbers of people. The bar could be set even higher as simple, routine tasks become automated.

I do believe that creative jobs are immune from automation, and certainly, there are no AI models that can write stories, so I still plan on being a writer in the long-term. We’ll have to wait and see whether my creative abilities end up being more lucrative than my technical ones.

Friday, July 8, 2022

Degrees, Money and the Future

For a long time, it was politically incorrect to protest the huge increase in university/college attendance across developed nations in Europe and North America. University attendance went from approximately 14% in the 1970s to over 40% today. Tony Blair famously wanted 50% of youngsters to attend university. A degree was lauded as a springboard to socio-economic mobility, and graduates were supposed to boost GDP growth and tax revenues by taking up high-skilled jobs.

But the reality is turning the dream into a nightmare. The sad truth is that the number of graduates exceeds the number of graduate jobs, and many graduates—particularly from some of the humanities and social sciences—end up under-employed. Furthermore, the burden of college debt acts like a vice on the economy. It doesn’t matter whether the debt is private debt (like in the US) or public debt (like in most European countries). Most obviously, debt harms consumption. It has knock-on social effects on fertility and demographics; young people burdened with debt start a family later (or don’t start it at all), and struggle to afford a house. Moreover, from a macroeconomic perspective, the loans are dangerous, since a lot of them are bad—that is to say, some graduates will repay the full amount, or repay the interest, or repay within the expected maturity period.

The hard truth, that young people, parents, and policymakers don’t want to hear, is that high-skilled, university-level jobs make up at best 20% of the available jobs. I am also highly skeptical that this will increase in the future. The percentage may increase, but only because the total number of jobs will decrease as more jobs are destroyed by automation (high-skilled jobs are much more resistant to the effect of automation). The jobs that require a degree are highly technical and/or vocational: medicine, pharma, nursing, dentistry; programming; engineering; data science; and so on.

Automation, AI and the post-scarcity economy are things that have been discussed elsewhere in more detail than I will go into here. The long and short of it is: we are heading towards are a post-scarcity economy. We are not there yet, and the process will take time. Also, let me be clear that a post-scarcity economy does not mean the end of scarcity; some goods and services will remain scarce, but the majority will be available in abundance. To give you an idea: food, energy and consumer goods will be abundant. Heck, they are already pretty abundant right now. Have you ever seen a shortage of nails? Nails, like many other goods, can be mass-produced for a very low price, and they require almost no human input in their creation. Other things will remain more scarce, especially things that require a lot of high-skill human labour.

That’s the key, the salient point of this little essay: labour. Historically, we have regarded labour, work, as something valuable and worthy. An entire religion, Protestantism, revolves around the value of work. Work means steady payment; a livelihood. But labour is inherently tied to scarcity. Humans have to labour because things are scarce: food, shelter, medicine and so on. So what happens when all the work is done by robots and software? Let’s assume that everything could be done by robots or computers, just for the purposes of this thought experiment.

Ideally, we would live in a utopia; humanity would never want for anything. In reality, the barrier lying between us and this utopian vision is capitalism. We admire capitalism because it has worked well for the last two centuries or so—with government regulation and the managing of natural monopolies, of course. Adam Smith was more or less right. He was misunderstood by the braying free market neoliberals; Adam Smith never argued that the “invisible hand” would make markets self-regulating in general (only if a very specific set of criteria were fulfilled). A mixed market+command economy is the way to go. This is something that France, the UK, Scandinavia and Germany understood, but which the US and Soviet Union did not.

Unfortunately, I think the economic model of the past century may no longer function in this post-scarcity future, because it is still predicated on the idea of renumeration for labour. To put it bluntly: in the future, many, perhaps most, of the population will be unemployed. They will be unemployed for no other reason than that they will have nothing to do.

I believe that we will need a Universal Basic Income at some point in the not-too-distant future. Moreover, I think a successful economic model of the future will still have capitalism (i.e. capital, companies, competition etc.) but in a far more diminished way. Capitalism will be limited to areas of rapid innovation, scarcity, and high differentiation. Everything else will be administrated by organs of the state, be it local, regional or national governments. Money will still exist, so this will be socialism, not communism. The state already administers 40–50% of the economy measured by GDP, so this should not be a big pill to swallow (newsflash, Americans). Why will it be adminstered by the state, you ask?

This leads onto the next point. Capitalism is unstable and destructive. It creates a small number of winners and a large number of losers. (Sorry, right-wing Americans.) In the future, this tendency will be exaggerated until society will fray apart. An industrial reserve army of the unemployed—an army of losers—will be created. There are only three possible scenarios. One, capitalism tries to maintain itself through coercion. Two, socialism prevails. Or three, massive social unrest results in anarchy, and the post-scarcity economy is destroyed, bringing us back to the status quo ante… by which I mean something before 1770: the medieval world. This scenario seems more likely in third world countries that are politically unstable.

Let’s go back to square one, and the original point of this essay. More university degrees will not lead to better pay or employment. There are powerful economic and technological forces at play that are leading us to a world with high unemployment. Instead of creating more debt and broken dreams, policymakers need to focus on managing the transition to a post-scarcity economy based on UBI, the provision of basic goods, and reduced inequality.

What does this look like, in practical terms? Well, it won’t be a world of perfect equality. I don’t believe such a thing is possible or even desirable—and I say this as a staunch socialist. Natural inequality is the reality of the world we live in. Some humans are smarter than others, or more talented, or hardworking; they should be rewarded. So the economic situation will look like this. Everyone will be guaranteed a basic income of, say, 2000 euro a month. They can earn extra by participating in the labour market when possible. Some people will earn high salaries, like 3K, 4K, 5K a month because they do something difficult (and they will pay taxes on their income). And there will be a few millionaires or billionaires, but fewer than there are today. The state’s revenues will shift from being predominantly taxation funded to being funded more by the sale of goods, e.g. food and housing. This is because there won’t be enough taxpayers to fund public goods like healthcare and (obviously) UBI itself otherwise.

Money is hard to understand for the layperson. Money is not scarce; it can be created at will. But scarce things do have a higher price. This is why creating more money causes inflation: there is more money, but the number of goods does not increase.

How can the state fund UBI? It’s not through taxation: you can tax UBI, but since that money comes from the government, there is no mathematical way to fund UBI with itself. Rather, the state can simply print money and ensure that the supply of goods (which it controls) matches up. The state will also tax corporations that make large amounts of money through the sale of goods and services. This system will ensure that basic goods are provided, but also that people have disposable income to spend on more whimsical things—art, fine dining, holidays, whatever—at their discretion. Remember that this will not be a truly post-scarcity society, just a “mostly” post-scarcity one. Provided that the money supply is managed sensibly, this system will work very well.

Now that I think about it, this system could already be partially implemented in the world we live in today, since we are already approaching post-scarcity in some ways. But to accomplish this, we have to have political awareness. We cannot allow our politicians to further the interests of the rich, and pull the wool over our eyes. People are not unemployed because they are lazy, or because they don’t have a degree, or because they don’t know how to code. Only 1% of the population knows how to code, and demand is at most 2 or 3%. People are unemployed because of technological growth.

Some people will not like what I am saying about degrees. Humanities people in particular think that humanities degrees are being devalued in favour of STEM. But actually my argument has nothing to do with funding one or the other. It is true that humanities degrees do not pay as well as (most) STEM degrees, so that is a good argument to avoid going into debt for one. But I would very much be in favour of subsiding degrees so that the best humanities students can study for free. The real problem—which I want you, the reader, to understand—is when we make 1 in 2 youngsters get a degree just for the sake of it. Believe me, all those psychology students (psychology is one of the most popular majors) don’t really want to be doing a degree; they just want to party. Them getting a degree, however, prejudices bright students who are genuinely interested in learning. It strains financial resources and devalues the worth of a degree.

Thanks for reading this far. If you have some thoughts to share, please comment below.

Monday, June 27, 2022

Guide: Setting up zswap

This is a guide to setting up zswap on Linux-based operating systems. What is zswap and why should you use it? Zswap is a way of giving your computer additional virtual memory, like increasing your RAM. It is very useful for systems with low RAM (< 8 GB) and quite useful even for systems with more RAM. This guide will show you how to setup zswap with the lz4 compression algorithm, which is very fast. But first, to answer some questions.

Warning for BTRFS users: read the wiki before creating a swapfile. It is easier to use ZRAM instead.

Does this come with a performance penalty?

There is no such thing as free lunch – compressing and decompressing pages in virtual memory will tax your CPU. However, using compressed memory is faster than swapping to an SSD, and orders of magnitude faster than swapping to a spinning hard disk. It is also better than running out of memory, which results in either your system locking up or the out-of-memory killer killing some important process. As oom killers are not very intelligent, it is wise to avoid this.

Note: there is no performance penalty until you actually start swapping, just so we’re clear.

What about ZRAM?

ZRAM is good too; I have used it. But zswap does not compress pages which are incompressible, instead sending them to your swap file. This is a good thing, as it avoids wasting CPU cycles compressing pages that are not compressible anyway. ZRAM is good for hard disk or SD-backed computers; I feel zswap is more appropriate for SSD-backed devices as swapping to an SSD is not such a big problem.

Step 1

Note: run all the commands below in the terminal, copying them one line at a time and hitting enter. Make sure they are copied correctly.

First check if you have a swap file by running free -h. If you do have a swap file, continue to the next step. Otherwise run the code below.

sudo su
fallocate -l 4G /swapfile
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile
echo '/swapfile none swap sw 0 0' | tee -a /etc/fstab

Step 2

Note: the following assumes are you are using grub. PopOS users should follow the instructions here.

Run sudo nano /etc/default/grub and edit the line GRUB_CMDLINE_LINUX_DEFAULT to read:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash zswap.enabled=1 zswap.compressor=lz4 zswap.max_pool_percent=50 zswap.zpool=z3fold"

What does the max pool percent variable mean? This refers to the maximum % of your RAM that will be taken up with compressed storage. It is dynamically allocated, so it doesn’t take up any space until you actually start using it. For most systems, 50% is a good maximum. For really low memory systems, you can try 70%. Anything higher will make the system unusably slow (Google has actually benchmarked this for Chrome OS).

Save your changes (type Ctrl+X and type y and then enter). Now run:

sudo update-grub

Step 3

Run the following:

sudo su
echo lz4 >> /etc/initramfs-tools/modules
echo lz4_compress >> /etc/initramfs-tools/modules
echo z3fold >> /etc/initramfs-tools/modules
update-initramfs -u

You are done! Reboot and run cat /sys/module/zswap/parameters/enabled. If zswap is working, you should see a Y printed.

Sources

LKML

Linux Kernel Documentation

Make Tech Easier

IBM

Guide: Using gpu-screen-recorder with your NVIDIA GPU

GPU screen recorder is an awesome open source program that allows you to record a window or screen using NVENC, a technology specific to Nvidia cards that is used to encode video without using the CPU. This is extremely useful for any low-spec PC, and you would otherwise need a very powerful multicore processor to be able to game and record at the same time – especially if you game at 4K. Furthermore, this tool records in h.265, a very bandwidth efficient compression algorithm that does wonders for your disk space.

Why not OBS?

Gpu screen recorder uses less CPU, and does not harm your fps, unlike OBS. It also feels smoother, with less stuttering. This is because OBS has to copy frames between the host and device; whereas GSR uses CUDA and runs entirely on the GPU. Furthermore, I’ve experienced a lot of difficulty getting OBS to record in h.265 (I believe OBS depends on ffmpeg which may not be packaged correctly for your distribution).

Is Wayland supported?

Not as far as I know. I attempted to run it on the Plasma Wayland session but was unable to get it to work.

Does it work with PRIME?

Yes, but it can only record a window. Make sure your game is running in borderless window mode instead of true fullscreen.

Installation guide

If you are on Arch, install the AUR package of the same name. Else, clone the git repository with git clone https://repo.dec05eba.com/gpu-screen-recorder. Make sure to install git if it isn’t already installed (sudo apt install git on Ubuntu). Also install xdotool if it is not already installed, using your package manager.

Change to the newly created directory. If you are an Ubuntu user, run sudo ./install_ubuntu.sh Otherwise you will have to install the dependencies yourself and compile the program.

Now, go to this Github link and select Code -> Download ZIP. PRIME users, SKIP this step. Extract the archive, then run sudo ./patch-fbc.sh.

Usage

To record the whole screen:

gpu-screen-recorder -w "screen" -c mp4 -f 60 -a "$(pactl get-default-sink).monitor" -o test_video.mp4

You may replace test_video.mp4 with a more useful name.

To record a window:

gpu-screen-recorder -w $(xdotool selectwindow) -c mp4 -f 60 -a "$(pactl get-default-sink).monitor" -o test_video.mp4

Make sure to start the game beforehand and select its window by clicking on it. Note: xdotool can only see XWayland windows, not native Wayland windows. And I haven’t tested it with Gamescope yet.

Why does this guide exist?

Because documentation is scarce and it took me a lot of time to setup. Also, a lot of people don’t know this tool exists, and may be struggling with OBS.

Monday, June 20, 2022

The Problem with Manjaro/Arch

I have been using Manjaro for about a year. My initial love for Manjaro came about because of three things. One, it made the installation of the Nvidia driver as painless as possible – by allowing you to boot directly using the binary driver and avoiding the need to mess with nouveau and nomodeset (looking at you, Fedora).

Two, Manjaro ships the latest Linux software in its repositories. This is a welcome change from Ubuntu or Debian; the packages in the repositories can be badly out of a date on an Ubuntu LTS, and even the latest Ubuntu might run a version or two behind. Related to this, I am shocked by how many packages in the Ubuntu repositories just don’t work. For example: Back In Time, a popular backup tool, does not work on the latest Ubuntu 22.04.

Thirdly, Manjaro also ships the latest releases of various desktop environments, allowing the user to try out different UIs if they don’t like the one they started with. (Or because they wanted to find out which DE actually supports fractional scaling properly… signed, yours truly.)

Alas, it was not to be. A few days ago, the Manjaro install on my Nvidia-powered desktop PC imploded after I upgraded using pamac. I do not mean to say that something was broken after an update; I mean to say that the pamac froze halfway through, and rendered the entire system unrecoverable. Firefox stopped working. KDE wouldn’t respond to mouse and keyboard input. After a force restart, I was left with an unbootable system with no kernel.

This tells us a few things. Firstly, pamac is a horrible program. But these were not the only issues that troubled my Manjaro experience. For example, GPG signature errors are depressingly common on both Arch Linux and Manjaro.

Another problem, inherent to the very design of pacman/Arch Linux, is that you cannot perform partial upgrades safely. Need to grab a small package/program in order to accomplish a task? You may well have a multi-gigabyte update that needs to be applied first, with a real risk of bricking your system, as I found out when I innocently tried to install a 100KB package.

Furthermore, if the update command pacman -Syu fails, it will not fail gracefully like on, say, Fedora, where dnf will rollback the transaction. No, it will leave your system in the before-mentioned partially upgraded state. From the Arch Wiki:

Note that if pacman -Syu does not perform the upgrade because of an error, the end result is the same as running pacman -Sy. Therefore, the error must be resolved and the upgrade operation completed as soon as possible.

Yikes!

In fact, the more I read about pacman, the less I like it. Common operations lack human-readable names, like -Syu instead of dnf update or apt’s update && upgrade. Pacman is not designed to keep multiple versions of a library installed, which causes conflicts if program X needs library version 0.12 but program Y needs version 0.13. (It is possible to fix this manually with various solutions like LD_LIBRARY_PATH, symlinks, chroots etc, but nothing user-friendly.) Contrast this to Flatpak, which can not only keep multiple runtimes installed, like org.Gnome.Platform 41 and 42—but deduplicate them as well.

It is my opinion that these technical problems render the very idea of a user-friendly Arch distribution like Manjaro or Endeavour laughable. Updating is just too dangerous, especially since many users like to install a lot of programs. I am guilty of this as well.

The AUR

There are three problems with the AUR. Firstly, it is insecure because anyone can publish a package there. Secondly, a lot of AUR software is distributed in source form, which takes ages to compile — and might fail. The third Manjaro-specific problem is that since Manjaro holds packages back for two weeks, this can cause AUR programs to fail because the system is not up-to-date enough.

The sad reality is that a lot of software is missing from the Arch repositories and the AUR is the only convenient way to get this software. Who wants to fiddle around with alien or compile software manually?

So what’s the alternative?

Flatpak is the best package manager I have ever used. It will never brick your system (it doesn’t mess with system files); it handles dependency conflicts and deduplication beautifully; it just works! About the only bad things I can say are that sometimes, I have to use Flatseal to change permissions, and there are some small theming issues.

But not every program can be distributed as a self-contained Flatpak. So what Linux should I use for the base system?

Currently, I am using Kubuntu on the desktop because it distributes a recent kernel, allows easy installation of the Nvidia driver, and has reasonably up-to-date packages. But some packages are still out of date—KDE software, ironically enough. I also think the apt package manager could be improved. Maybe I should try aptitude.

But is there really an alternative to the Ubuntu/Mint/Pop world?

  • Fedora KDE: The live session froze before it even got to the installer. The problem is that they made Wayland the default on KDE, which does not work with the nouveau drivers. (Fedora refuses to ship the Nvidia binary drivers or patent-encumbered codecs.) This does not inspire confidence in the level of QA and testing that went into the distro.
  • OpenSUSE Leap: too out of date.
  • OpenSUSE Tumbleweed: I’ve read mixed reviews on this one. It pushes updates really, really frequently. No human QA goes into it, only automated QA – and even a really good automated system like openQA can’t replace humans. Arch at least has humans testing its packages. Even with its fancy btrfs snapshots, I am not adventurous enough to try it.

The only alternative to Ubuntu I would consider is a Fedora with sane defaults. But then, I guess there’s a reason Ubuntu and its deriatives have been more popular than Fedora and Arch for a very long time.

EVs are not the future—hybrids are

There has been a wild surge in optimism in EVs—really, a kind of hysteria—with the EU and UK governments hoping to ban combustion engines in...