Perl improvements - going from 5.14 to 5.24

In the 5 major versions between 5.14 and 5.24, a few things changed. Here's a lightly edited version of the summary we sent out internally as preparation for the upgrade.

Note that Perl5 is currently on 5.28, so this list is already out of date. The exciting new features you've been waiting for may already have been implemented - have a look at perldelta to find out the latest and greatest!

Key/value slices

%x{qw(first second)} provides the 4-element list:

• 'first'
• $x{first}
• 'second'
• $x{second}

This means that code like this:

(exists $x{first}  ? first  => $x{first} : ()),
(exists $x{second} ? second => $x{second} : ()),

can now be written as

%x{grep exists $x{$_}, qw(first second)}

Postfix derefs

Instead of @{$some->object->that->eventually->returns->an->arrayref->after->many->chained->method->calls}, you can put ->@* at the end, as $some->object->...->chained->method->calls->@*. Same for other sigils:

• ->$*
• ->@*
• ->%*
• ->**

... maybe even ->&* too.

Autoderef has gone - so keys $hashref or values $arrayref are no longer available. Use keys %$hashref or keys $hashref->%* instead.

Internal variables

$$ is writable and can no longer be trusted...

... but other not-quite-constants such as undef are now hardcoded readonly, so popular idioms like this will break:

$ perl -le'my $x = \undef; &Internals::SvREADONLY($x => 0); $$x = 3; print undef'
3

$[ goes away, so all arrays now start at zero.

The filehandle used by $. or tell/eof can now be accessed via ${^LAST_FH}.

Debugger

A few smaller fixes, but for me the most useful one is that

b Some/File.pm:123

sets a breakpoint on a specific line in a file. Previously you'd have to do this in two steps by switching the file with f then setting the breakpoint.

Experimental things

See perldoc perlexperiment for full details.

smartmatch

Smartmatch (~~/given/when) - just avoid, this has been a disaster. If you need a switch statement, Switch::Plain is perhaps the cleanest option for now. For "is $x in @y"-type queries, try Data::Munge::elem, or an explicit codeblock with List::Util::first/firstidx.

Constant subs

use experimental qw(const_attr);
use List::Util qw(sum0);
my @points = (...);
my $volatility = sub:const {
 sqrt(sum0 map exp(-2 * $_ ** 2), @points)
};

Executes the coderef immediately, and returns a new constant sub.

current_sub

The current CV is accessible via __SUB__:

use experimental qw(current_sub);
use Future; # only needed for this example
my @pending = (...);
my $code = sub {
 return Future->done unless defined(my $item = shift @pending);
 $item->()->then(__SUB__)
};

Lexical subs

use experimental qw(lexical_subs);
{ my sub example { ... } }

These are new, experimental and sometimes useful for tests or simplifying logic, but overall best treated with caution.

Sub signatures

Yes, we finally have sub xyz($self, $first, @params) { } support. Still experimental, but so far seems to work pretty well.

Refaliasing

my $y = "example";
\$x = \$y, $y = "changed";
print $x

Works for most things, including package/local/state vars.

Bitwise string operators

&|^~ now have explicit string versions. Of course, instead of following the usual Perl tradition of using symbols for numeric ops and words for string ops, they use a . suffix instead.

$ perl -le'use experimental qw(bitwise); print "this" ^. "    "'
THIS

Parser changes

• You can now write FILE, LINE etc. with trailing (), e.g. FILE().
• Minor changes to prototypes - \$ and _
• for qw(x y z) { } no longer works, you have to include the extra () now - perl 5.14 used to warn about this anyway
• $<control char> no longer works, although you can still use ${<control char>} if you're feeling particularly masochistic.
• ?? is no more.
• split / / is now equivalent to split ' ', so it's effectively the same as split /\s+/ with the side effect of also throwing away leading/trailing whitespace
• use and require will stop on the first file in @INC, so if there's something with insufficient permissions or other errors, you'll get an exception rather than falling back to the next entry in @INC.
• You'll get a warning when you write statements like this:

return $obj->method or die 'error';

since the 'or die' clause would never have been executed.

• pack() and x can now be constant-folded.
• DESTROY with first statement as return (after peephole optimisations) will be elided completely.
• $& no longer hurts performance for every other regex op in the rest of the process. This means use English; no longer slows down your program.
• Lexical $_ has gone

Performance

my ($x, $y, $z) = @_;

is now a single op, so it can benchmark faster than individual shift()s. This is a micro-optimisation, but sometimes of use in hot codepaths. Might be worth considering if your normal sub/method definitions consist of many shifted parameters, but as always benchmark+profile before rewriting your entire codebase.

Probably many more things too.

Security

Several things here, including the new hashing algorithm and randomisation.

• One major change that has caused much discussion was about how to avoid the longstanding default of including . in PERL5LIB. Expect to see a bit of boilerplate code appearing which does the equivalent of { local @INC = @INC; pop @INC if $INC[-1] eq '.'; require ...; }.
• double-diamond fixes the perl -ne'...' "|rm" auto-execute case: if you use <<>> instead of <>, a 3-arg open is used instead, so the ARGV entry is treated as a filename with somewhat lessened risk of shell expansions.
• Locale::Maketext finally fixes the arbitrary code execution vulnerability, but it's still not perfect (translations can still trigger calls to Local::Maketext methods).
• Lots of fuzztesting (mostly using afl) since 5.18, so there are many edge cases that have been patched over the last few releases.

XS

Several changes affect old XS code, but copy-on-write strings are perhaps the most likely to cause problems.

• use is_utf8_char_buf instead of is_utf8_char
• try to avoid the old macros and offsets for accessing pads (there's a basic API for accessing that information now)... documentation is still mostly nonexistent for all this, but perlxs has had a few updates.
• You may also need to look at the XS_EXTERNAL() macro, since XSUBs are declared static by default now.
• Unused array slots hold NULL rather than Perl undef, so any code that does an av_store with PL_sv_undef will most likely not work as expected (the slot will be readonly).
• not really XS, but: there's a C backtrace facility now, see perlhacktips for details. You'll probably need a custom compile to try it out.

Unicode

Various Unicode things are fixed, including variable/method/function names, eval and additional features in charnames. Still doesn't work cleanly on the commandline, but it's a step or two closer to full internal Unicode support in scripts and modules.

5.24 supports the Unicode 8.0 standard - 9.0 was imported in the 5.25 development series.

\N{BELL} is now a phone emoji, \N{BEL} if you want a longer way to type \a.

Bugfixes

Many and varied. Here's a couple of examples:

• Sometimes old entries would be visible in pads in recursive DESTROY calls, this has mostly been fixed. See RT#119311 for example. Somewhat obscure, but it always used to be one of the few places where my @x = (); was required instead of just my @x;.
• do { ... } until 1; doesn't crash so much.

Miscellaneous changes

• Characters quoted by quotemeta() have changed (specifically, a few extra Unicode codepoints are quoted - the ASCII range is unaffected).
• Hashing has changed (several times), we now use the ONE_AT_A_TIME_HARD hashing algorithm by default. There are some environment variables to control hashing behaviour - see PERL_PERTURB_KEYS in perldoc perlrun, and the Hash::Util module. The changes are mostly in the name of security and performance.
• \s in regex now matches (most) whitespace, including vertical tab - so [\s] is probably equivalent to [\h\v]. Knowing Unicode, they've introduced even more orthogonal directions by now, so whitespace may actually cover more than just horizontal/vertical
• substr offset+length follow changes to original string
• tied returns the value
• we now have fc for casefolding, generally tends to be a better option than lc/uc since it's more consistent with situations such as ß <-> SS or ς <-> σ.

Threads

Perl doesn't have threads and any attempt to convince me otherwise will be met with polite disbelief.