Category: News
parallel @ Savannah: GNU Parallel 20220722 (‘Roe vs Wade’) released
GNU Parallel 20220722 (‘Roe vs Wade’) has been released. It is available for download at: lbry://@GnuParallel:4
Quote of the month:
The syntax for GNU Parallel is so slick that I often use it just to make my script read nicer, and the parallelism is a cherry on top.
— Epistaxis@reddit
New in this release:
- –colour-failed will color output red if the job fails.
- sql: support for InfluxDB.
- Polarhome.com is dead, so these OSs are no longer supported: AIX HPUX IRIX Minix OPENSTEP OpenIndiana OpenServer QNX Solaris Syllable Tru64 Ultrix UnixWare.
- Bug fixes and man page updates.
News about GNU Parallel:
- GNU Parallel used in “Hitting the Target” https://www.centrefornetzero.org/wp-content/uploads/2022/05/ABM-Report-Final.pdf
GNU Parallel – For people who live life in the parallel lane.
If you like GNU Parallel record a video testimonial: Say who you are, what you use GNU Parallel for, how it helps you, and what you like most about it. Include a command that uses GNU Parallel if you feel like it.
About GNU Parallel
GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.
If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.
GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.
For example you can run this to convert all jpeg files into png and gif files and have a progress bar:
parallel –bar convert {1} {1.}.{2} ::: *.jpg ::: png gif
Or you can generate big, medium, and small thumbnails of all jpeg files in sub dirs:
find . -name ‘*.jpg’ |
parallel convert -geometry {2} {1} {1//}/thumb{2}_{1/} :::: – ::: 50 100 200
You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/
You can install GNU Parallel in just 10 seconds with:
$ (wget -O – pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ ||
fetch -o – http://pi.dk/3 ) > install.sh
$ sha1sum install.sh | grep 883c667e01eed62f975ad28b6d50e22a
12345678 883c667e 01eed62f 975ad28b 6d50e22a
$ md5sum install.sh | grep cc21b4c943fd03e93ae1ae49e28573c0
cc21b4c9 43fd03e9 3ae1ae49 e28573c0
$ sha512sum install.sh | grep ec113b49a54e705f86d51e784ebced224fdff3f52
79945d9d 250b42a4 2067bb00 99da012e c113b49a 54e705f8 6d51e784 ebced224
fdff3f52 ca588d64 e75f6033 61bd543f d631f592 2f87ceb2 ab034149 6df84a35
$ bash install.sh
Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.
When using programs that use GNU Parallel to process data for publication please cite:
O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014.
If you like GNU Parallel:
- Give a demo at your local user group/team/colleagues
- Post the intro videos on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
- Get the merchandise https://gnuparallel.threadless.com/designs/gnu-parallel
- Request or write a review for your favourite blog or magazine
- Request or build a package for your favourite distribution (if it is not already there)
- Invite me for your next conference
If you use programs that use GNU Parallel for research:
- Please cite GNU Parallel in you publications (use –citation)
If GNU Parallel saves you money:
- (Have your company) donate to FSF https://my.fsf.org/donate/
About GNU SQL
GNU sql aims to give a simple, unified interface for accessing databases through all the different databases’ command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.
The database is addressed using a DBURL. If commands are left out you will get that database’s interactive shell.
When using GNU SQL for a publication please cite:
O. Tange (2011): GNU SQL – A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.
About GNU Niceload
GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.
The Windows 11 taskbar is getting better for people who open tons of apps
Peoples BLOG: Usage of Local Php Security Checker for Drupal Applications
Linux skills: 9 tutorials to get more from your text editor
Are you getting everything you need out of your text editor? Read Enable Sysadmin’s recent articles about Linux text editors to find out what you’re missing. Read More at Enable Sysadmin
The post Linux skills: 9 tutorials to get more from your text editor appeared first on Linux.com.
Firefox Presents: A nerdcore rap artist defying expectations
As far as music subgenres go, nerdcore hip-hop wouldn’t exist in its current form without the internet. Those who create it — self-proclaimed nerds who rap about video games, science fiction or just everyday life as a misfit — record music at home, collaborate via web file transfers and find cheerleaders out of strangers online. […]
The post Firefox Presents: A nerdcore rap artist defying expectations appeared first on The Mozilla Blog.
datamash @ Savannah: GNU Datamash 1.8 released
This is to announce datamash-1.8, a new release.
Datamash is a command-line program which performs basic numeric, textual and
statistical operations on input textual data.
This is the first release for new maintainer Tim Rice, with much appreciation
to Shawn Wagner and Erik Auerswald for their help. See the AUTHORS and THANKS
files for additional credits and acknowledgements.
GNU Datamash home page:
https://www.gnu.org/software/datamash/
Please report any problem you may experience to the bug-datamash@gnu.org
mailing list.
Happy Hacking!
– Tim Rice
==================================================================
Here are the compressed sources and a GPG detached signature[*]:
https://ftp.gnu.org/gnu/datamash/datamash-1.8.tar.gz
https://ftp.gnu.org/gnu/datamash/datamash-1.8.tar.gz.sig
Use a mirror for higher download bandwidth:
https://ftpmirror.gnu.org/datamash/datamash-1.8.tar.gz
https://ftpmirror.gnu.org/datamash/datamash-1.8.tar.gz.sig
[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact. For instructions about how to do this, please
refer to https://ftp.gnu.org/README. (In particular you will need to
retrieve the GNU keyring rather than using any keyservers.)
==================================================================
The checksums of the archive are:
$ sha1sum datamash-1.8.tar.gz
e77e15ed2c6b17b4045251fd87f16430c3bf2166 datamash-1.8.tar.gz
$ sha256sum datamash-1.8.tar.gz
94a4e11819ad259aa3745b7eca392e385e3a676d276e8cbb616269dbbb17fe6d datamash-1.8.tar.gz
$ b2sum datamash-1.8.tar.gz
dfe4060ea65ea46a1796e01463fd9b0e55c2d633d06da153f585a3a569acf3e9211a14cb3905daf8ecae347358daa04db940d557b909f0ce5ebbba2f57d3a410 datamash-1.8.tar.gz
==================================================================
NEWS
- Noteworthy changes in release 1.8 (2022-07-23) [stable]
-
- Changes in Behavior
Schedule -f/–full combined with non-linewise operations for deprecation.
In a future release, -f/–full will only be usable with operations where
it makes sense. For now, we print a warning to stderr when -f/–full is
used with non-linewise operations, and such usage will no longer be
supported.
The bin operation now uses more intuitive bins. Previously, a command
such as `datamash bin 1 <<< -0` would output -100; and -100 did not fall
in its own bin. We now require all bins to take the form `[nx,(n+1)x)`
with integer n and bin width x. We discard the sign on -0 and gate such
inputs into the [0,x) bin.
Operations taking more than one argument now provide more complete output
with –header-out. Previously, an operation such as `pcov x:y` would
produce an output header like `pcov(y)`, discarding the `x`. The new
behavior will output header `pcov(x,y)`.
datamash(1) no longer ignores –output-delimiter with the rmdup operation.
-
- New Features
New datamash option –sort-cmd argument to specify the program used
by the -s option to sort input, plus enhancements to the security and
portability of building sort command lines.
New datamash option -c/–collapse-delimiter=X argument uses character
X instead of comma between values in collapse and unique lists.
New datamash operations: mean square (ms) and root mean square (rms).
Decorate now supports sorting IP addresses of both versions 4 and 6
together. IPv4 addresses are logically converted to IPv6 addresses,
either as IPv4-Mapped (ipv6v4map) or IPv4-Compatible (ipv6v4comp)
addresses.
Add two command aliases:
‘echo’ may now be used instead of ‘cut’.
‘uniq’ may now be used instead of ‘unique’.
-
- Improvements
Updated the bash completion script to reflect recent additions.
-
- Bug Fixes
Datamash now passes the -z/–zero-terminated flag to the sort(1) child
process when used with “–sort –zero-terminated”. Additionally,
if the system’s sort(1) does not support -z, datamash reports the error
and exits. Previously it would omit the “-z” when running sort(1),
resulting in incorrect results.
Documentation fixes and spelling corrections.
Incorrect format in a decorate(1) error breaking compilation on some
systems.
datamash(1), decorate(1): Fix some minor memory leaks.
datamash(1) no longer crashes when the unique or countunique operations
are used with input data containing NUL bytes. The problem was reported
in https://lists.gnu.org/archive/html/bug-datamash/2020-11/msg00001.html
by Catalin Patulea.
datamash(1) no longer crashes when crosstab with –header-in is called
by field name instead of index. I.e. `datamash –header-in ct x,y` now
works as expected.