On reading documentation when programs don't work as expected
No matter how often you use UNIX tools, once in a while you get caught out trying to put everything together. This happened this morning while I was setting up a cronjob and it didn't work as expected.
This happened to me earlier this week when I tried to setup a quick cron script to dump the contents of an SQL table every day. I had set it up as follows:
37 3 * * * /usr/lib/postgresql/8.2/bin/pg_dump db -U backup -t table | bzip2 - > /backups/table-monitoring/`date +%Y%m%d`.table.sql.bz2
I checked the next day that things were working and found the directory empty. Interesting.
After a quick check of my email I found the culprit:
/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /bin/sh: -c: line 1: syntax error: unexpected end of file
This is where things got complicated. Had I checked crontab(5) this story would probably have ended here. I didn't.
After a couple of attempts at escaping this command line to make things work I ended up with the following:
backup:~% ls /backups/table-monitoring | head -n 3 `date \+%Y%m%d`.table.sql.bz2 date +%Y%m%d.table.sql.bz2
Even more interesting...
Turns out the answer is quite simple. The man page has this to say about it:
The ‘‘sixth’’ field (the rest of the line) specifies the command to be run. The entire command portion of the line, up to a newline or % character, will be executed by /bin/sh or by the shell specified in the SHELL variable of the crontab file. Percent-signs (%) in the command, unless escaped with backslash (\), will be changed into newline characters, and all data after the first % will be sent to the command as standard input. There is no way to split a single command line onto multiple lines, like the shell’s trailing "\".
Ooops. A quick change later, I end up with the following snippet:
37 3 * * * /usr/lib/postgresql/8.2/bin/pg_dump db -U backup -t table | bzip2 - > /backups/table-monitoring/`date +\%Y\%m\%d`.table.sql.bz2
I'm still wondering about the usefulness of the % to send data to the standard input as opposed to using standard pipes though.
Developed by the CIO executive Council in conjunction with Rob Livingstone Advisory, Pathways Advanced is a 12-month CIO delivered, small group, mentor based professional leadership development program. Pathways Advanced brings together best practice, thought leadership and business insights for today’s most promising ICT professionals
- FTStudio Design ManagerVIC
- CCWeb / Mobile Developer - Magento - HTML5, CSS - Excellent CMS SkillsNSW
- FTDigital Performance Manager - MediaNSW
- FTAccount Manager - DataNSW
- FTDigital Marketing CoordinatorNSW
- FTSenior SEO ManagerNSW
- FTClient Service Director - Search: SEO & PPCNSW
- FTChief Information Officer - CSIROACT
A guide for security and network architects in designing, deploying, and managing architecture to protect against increasingly sophisticated, application-layer DDoS attacks.
- Sharp smartband LCD uses 1,000 times less power
- HP's move into 3D printing will radically change manufacturing
- Hungary ditches Internet tax plans after protests
- Sony's new mobile chief has hands full as handsets struggle
- IBM joins Tencent to target China's growing enterprise cloud market
- Samsung attacks Chinese rivals with new mid-range Galaxy phones
- FCC's Wheeler said to mull hybrid approach to net neutrality
- Some Aussie businesses using DevOps to improve customer engagement and reduce IT spend: report
- The Google shakeup continues: Andy Rubin is out
- Google project aims to preserve privacy when collecting software stats
- Major banks ready their own mobile payment apps
- Android creator Andy Rubin leaving Google
- Data retention is necessary red tape: Turnbull
- Zuckerberg to connect with regular folk in first 'community Q&A'
- In Pictures: 12 shocking social media horror stories