[Slackbuilds-users] slackrepo fails to build youtube-dl due to URL encoding

[Erich Ritz]

------- Original Message -------
On Friday, June 23rd, 2023 at 6:40 PM, Jeremy Hansen <jebrhansen+SBo at gmail.com> wrote:


> On Fri, Jun 23, 2023, 8:22 AM Erich Ritz via SlackBuilds-users <slackbuilds-users at slackbuilds.org> wrote:
> 
> > Hello Dimitris and Andrew and list,
> > 
> > I don't know if this message is more appropriate for the slackrepo maintainer or the youtube-dl maintainer.
> > 
> > When trying to build youtube-dl with slackrepo it fails with:
> > 
> > Building network/youtube-dl (update for version 2021.12.17+20230618_07af47960) 2023-06-23 14:25:18
> > Removing old source files ...
> > Downloading source files ...
> > Verifying source files ...
> > Running youtube-dl.SlackBuild ... ETA 14:26:??
> > env MAKEFLAGS='-j176 -l89' nice -n 5 bash ./youtube-dl.SlackBuild
> > tar: /tmp/slackrepo/etr/slackrepo.MOV0A0/slackbuild_youtube-dl/youtube-dl-2021.12.17+20230618_07af47960.tar.xz: Cannot open: No such file or directory
> > tar: Error is not recoverable: exiting now
> > network/youtube-dl: youtube-dl.SlackBuild failed (status 2) 14:25:20
> > Unmounting chroot ...
> > :-( network/youtube-dl FAILED )-:
> > 
> > slackrepo downloads the source file, but it is saved with a "%2B" instead of "+" in the filename. This is due to curl not doing URL decoding. Note these (partial) excerpts from curl's man page:
> > 
> > -J, --remote-header-name
> > There's no attempt to decode %-sequences (yet) in the provided file name,
> > so this option may provide you with rather unexpected file names.
> > -O, --remote-name
> > There is no URL decoding done on the file name. If it has %20 or other
> > URL encoded parts of the name, they will end up as-is as file name.
> > 
> > But I can see why slackrepo chooses to use curl instead of wget for sourceforge (as pointed out by a comment in srcfunctions.sh of slackrepo):
> > 
> > $ wget --content-disposition https://downloads.sourceforge.net/project/dslackw/src/youtube-dl-2021.12.17%2B20230618_07af47960.tar.xz2023-06-23 10:14:19 (1.09 MB/s) - ‘youtube-dl-2021.12.17+20230618_07af47960.tar.xz?viasf=1’ saved [1172376/1172376]
> > 
> > sourceforge "helpfully" tacks on "?viasf=1" to the filename.
> > 
> > What's the best way forward? If I remember correctly SBo policy allows for (if not encourages) the use of the "--content-disposition" flag for wget and similarly the "-J" flag for curl. However, these break the filename in this case. Should slackrepo do its own URL decoding when using curl? Should slackrepo use wget for sourceforge and rename the files (stripping off "?viasf=1")? Should SBo discourage URL encoding in filenames?
> > 
> > Erich
> 
> 
> Have you tried the download_basename pragma in your hintfile?
> 
> download_basename — provide a symlink for source downloads using the URL's basename, for the benefit of SlackBuilds that don't expect content-disposition to be respected.
> 
> Jeremy

Thank you!  I had to specify both PRAGMA and DOWNLOAD (use "+" in the download URL):

$ cat /var/lib/slackrepo/etr/hintfiles/youtube-dl.hint 
PRAGMA="download_basename"
DOWNLOAD="https://downloads.sourceforge.net/project/dslackw/src/youtube-dl-2021.12.17+20230618_07af47960.tar.xz"

Now I'll just have to remember to update the DOWNLOAD URL in my hintfile next time it is updated...

Maybe a new PRAGMA could be added to replace DOWNLOAD URLs with decoded versions?

Erich