This may make some people pull their hair out, but Iād love to hear some arguments. Iāve had the impression that people really donāt like bash, not from here, but just from people Iāve worked with.
There was a task at work where we wanted something thatāll run on a regular basis, and doesnāt do anything complex aside from reading from the database and sending the output to some web API. Pretty common these days.
I canāt think of a simpler scripting language to use than bash. Here are my reasons:
- Reading from the environment is easy, and so is falling back to some value; just do
${VAR:-fallback}; no need to write another if-statement to check for nullity. Wanna check if a variableās set to something expected?if [[ <test goes here> ]]; then <handle>; fi - Reading from arguments is also straightforward; instead of a
import os; os.args[1]in Python, you just do$1. - Sending a file via HTTP as part of an
application/x-www-form-urlencodedrequest is super easy withcurl. In most programming languages, youād have to manually open the file, read them into bytes, before putting it into your request for the http library that you need to import.curlalready does all that. - Need to read from a
curlresponse and itās JSON? Reach forjq. - Instead of having to set up a connection object/instance to your database, give
sqlite,psql,duckdbor whichever cli db client a connection string with your query and be on your way. - Shipping isā¦ fairly easy? Especially if docker is common in your infrastructure. Pull
Ubuntuordebianoralpine, install your dependencies through the package manager, and youāre good to go. If you stay within Linux and donāt have to deal with differences in bash and core utilities between different OSes (looking at you macOS), and assuming you tried to not to do anything too crazy and bring in necessary dependencies in the form of calling them, it should be fairly portable.
Sure, there can be security vulnerability concerns, but youād still have to deal with the same problems with your Pythons your Rubies etc.
For most bash gotchas, shellcheck does a great job at warning you about them, and telling how to address those gotchas.
There are probably a bunch of other considerations but I canāt think of them off the top of my head, but Iāve addressed a bunch before.
So whatās the dealeo? What am I missing that may not actually be addressable?
I just donāt think bash is good for maintaining the code, debugging, growing the code over time, adding automated tests, or exception handling
If you need anything that complex and that itās critical for, say, customers, or people doing things directly for customers, you probably shouldnāt use bash. Anything that needs to grow? Definitely not bash. Iām not saying bash is what you should use if you want it to grow into, say, a web server, but that itās good enough for small tasks that you donāt expect to grow in complexity.
I donāt think youāll get a lot of disagreement on that, here. As mention elsewhere, my team prefers bash for simple use cases (and as their bash-hating boss, I support and agree with how and when they use bash.)
But a bunch of us draw the line at database access.
Any database is going to throw a lot of weird shit at the bash script.
So, to me, a bash script has grown to unacceptable complexity on the first day that it accesses a database.
We have dozens of bash scripts running table cleanups and maintenece tasks on the db. In the last 20 years these scripts where more stable than the database itself (oracle -> mysql -> postgres).
But in all fairness they just call the cliclient with the appropiate sql and check for the response code, generating a trap.
Thatās a great point.
I post long enough responses already, so I didnāt want to get into resilience planning, but your example is a great highlight that thereās rarely hard and fast rules about what will work.
There certainly are use cases for bash calling database code that make sense.
I donāt actually worry much when itās something where the first response to any issue is to run it again in 15 minutes.
Itās cases where we might need to do forensic analysis that bash plus SQL has caused me headaches.
Yeah, if it feels like a transaction would be helpful, at least go for pl/sql and save yourself some pain. Bash is for system maintenance, not for business logic.
Heck, I wrote a whole monitoring system for a telephony switch with nothing more than bash and awk and it worked better than the shit from the manufacturer, including writing to the isdn cards for mobile messaging. But I wouldnāt do that again if I have an alternative.
That is such a good guiding principle. Iām gonna borrow that.
On one conference I heard saying: āThere is no such thing as temporary solution and there is no such thing as proof of conceptā. Itās an overexaguration of course but it has some truth to it - thereās a high chance that your āsmall changeā or PoC will be used for the next 20 years so write it as robust and resilient as possible and document it. In other words everything will be extended, everything will be maintained, everything will change hands.
So to your point - is bash production ready? Well, depends. Do you have it in git? Is it part of some automation pipeline? Is it properly documented? Do you by chance have some tests for it? Then yes, itās production ready.
If you just āwrite this quick script and run it in cronā then no. Because in 10 years people will pull their hair screaming āwhat the hell is hapenning?!ā
Edit: or worse, theyāll scream it during the next incident thatāll happen at 2 AM on Sunday
I find it disingenuous to blame it on the choice of bash being bad when goalposts are moved. Solutions can be temporary as long as goalposts arenāt being moved. Once the goalpost is moved, you have to re-evaluate whether your solution is still sufficient to meet new needs. If literally everything under the sun and out of it needs to be written in a robust manner to accommodate moving goalposts, by that definition, nothing will ever be sufficient, unless, well, weāve come to a point where a human request by words can immediately be compiled into machine instructions to do exactly what theyāve asked for, without loss of intention.
That said, as engineers, I believe itās our responsibility to identify and highlight severe failure cases given a solution and its management, and it is up to the stakeholders to accept those risks. If you need something running at 2am in the morning, and a failure of that process would require human intervention, then maybe you should consider not running it at 2am, or pick a language with more guardrails.