"sorttable_customkey" attribute does not work

See the subject.

Oh, I see, you intentionally moved to "data-st-key" to conform with HTML5. Sorry.

But I still think you should add a workaround for backwards compatibility. Somehting like:

var oldattrib = node.getAttribute('sorttable_customkey');
if (oldattrib)
    return oldattrib;

[White-Tiger]

I didn't add backward compatibility for performance mainly. Also why should one intentionally generate or keep invalid HTML?
Can you think of a reason why a simple search&replace might not work? (which isn't just a mere fix for sorttable, but also reduces the size and validates)

But yes, it needs to be made more clear to which compatibility changes were made and probably how to adapt to those.

P.S. a custom key can often be replaced by custom sort functions. So you might consider to write your own which also drastically reduces the code size (as you won't need a custom key)
like:

    <script>
    /* new sort function */
    sorttable.sort_duration = function(a,b){
        var regex = /^(?:(\d+)\s?d\s?)?(?:(\d+)\s?h\s?)?(?:(\d+)\s?m\s?)?(?:(\d+)\s?s?)?$/;
        var tsA=0,tsB=0, match;
        match = regex.exec(a[0]);
        if (match) tsA = (match[1]<<0)*86400 + (match[2]<<0)*3600 + (match[3]<<0)*60 + (match[4]<<0);
        match = regex.exec(b[0]);
        if (match) tsB = (match[1]<<0)*86400 + (match[2]<<0)*3600 + (match[3]<<0)*60 + (match[4]<<0);
        return tsA - tsB;
    };</script>
    <table class="sortable" id="custom">
    <thead>
        <tr><th class="sorttable_duration">duration</th></tr>
    </thead>

the only downside: it might require more CPU resources from the client who wants to sort... depending on your actual table content. But you won't have to generate those on the server each time.

[twentyfortysix]

the data-st-key info is missing in https://www.kryogenix.org/code/browser/sorttable/#customkeys