  
  [1X9 [33X[0;0YWeb Downloads[133X[101X
  
  [33X[0;0YThe  [10XDownload[110X  operation  has been written by Thomas Breuer, incorporating a
  number  of  suggestions  from  Max  Horn,  for  version  0.77  of  [5XUtils[105X. It
  implements  downloading  a  file  from  within  [5XGAP[105X.  It  can  use the [5XIO[105X or
  [5XcurlInterface[105X  packages,  or  [13Xwget[113X  or  [13Xcurl[113X,  if  installed,  and it can be
  extended  with  other  download  methods quite easily. It is envisaged that,
  once  other  packages  have  started  to  use it, and any problems have been
  addressed, that the functions will be transferred to the main [5XGAP[105X library.[133X
  
  
  [1X9.1 [33X[0;0YFunctions for downloading files from the web[133X[101X
  
  [1X9.1-1 Download[101X
  
  [33X[1;0Y[29X[2XDownload[102X( [3Xurl[103X[, [3Xopt[103X] ) [32X function[133X
  
  [33X[0;0YThis  function  downloads the file with the web address [3Xurl[103X, which must be a
  string.[133X
  
  [33X[0;0YThe  result is a record which has at least the component [10Xsuccess[110X, with value
  [9Xtrue[109X if the download was successful and [9Xfalse[109X otherwise. In the former case,
  the  component  [10Xresult[110X  is  bound, whose value is a string that contains the
  contents  of the downloaded file. In the latter case, the component [10Xerror[110X is
  bound, whose value is a string that describes the problem.[133X
  
  [33X[0;0YThe  function  calls  the methods stored in the global list [10XDownload_Methods[110X
  until  one  of  them is successful. Currently there are methods based on the
  [5XGAP[105X  functions  [2XDownloadURL[102X  ([14Xcurl:  DownloadURL[114X) and [2XSingleHTTPRequest[102X ([14XIO:
  SingleHTTPRequest[114X),  and  methods  based  on  the external programs [10Xwget[110X and
  [10Xcurl[110X.[133X
  
  [33X[0;0YAn optional record [3Xopt[103X can be given. The following components are supported.[133X
  
  [8X[10XmaxTime[110X[8X[108X
        [33X[0;6YIf  this  component  is  bound  then  its  value must be a nonnegative
        integer [22Xn[122X, meaning that the function gives up after [22Xn[122X seconds.[133X
  
        [33X[0;6YA  zero value of [22Xn[122X means that no timeout is set, the method will never
        give up in this case.[133X
  
        [33X[0;6YThe  default  for  [22Xn[122X  is  given  by  the  value of the user preference
        [10XDownloadMaxTime[110X (see [14X9.1-3[114X).[133X
  
  [8X[10Xtarget[110X[8X[108X
        [33X[0;6YIf  this  component is bound then its value must be a string that is a
        local  filename,  and  the  function writes the downloaded contents to
        this  file;  the  returned  record does not have a [10Xresult[110X component in
        this case.[133X
  
        [33X[0;6YIf  the  download fails then a file created by the attempt is not left
        behind, unless [10Xresume[110X is set. A file that was already there before the
        call  is not removed, since it is not ours to delete – but note that a
        failed  attempt  may well have overwritten it, so its old contents are
        not guaranteed either.[133X
  
  [8X[10Xresume[110X[8X[108X
        [33X[0;6YIf  this  component  is  bound  and  has the value [9Xtrue[109X, and [10Xtarget[110X is
        given,  then  a  partially  downloaded  file  is continued rather than
        fetched again, and it is kept if the download fails again.[133X
  
        [33X[0;6YMethods  that  cannot  resume  decline  the  request, since they would
        discard  the  partial  file  that  a  method  which  can resume needs;
        currently  the  methods  based  on the external programs [10Xcurl[110X and [10Xwget[110X
        resume.  If  none  is available the download fails, and the caller can
        retry without [10Xresume[110X.[133X
  
        [33X[0;6YIt  is the caller's responsibility that an existing [10Xtarget[110X really is a
        prefix of what [3Xurl[103X delivers; otherwise the two get concatenated.[133X
  
        [33X[0;6YIf  the  server  does not support range requests then no data is lost:
        [10Xwget[110X  fetches the file again from the start, and [10Xcurl[110X fails and leaves
        the partial file alone.[133X
  
  [8X[10XverifyCert[110X[8X[108X
        [33X[0;6YIf this component is bound and has the value [9Xfalse[109X then those download
        methods  that  are  based  on  [10Xcurl[110X or [10Xwget[110X will omit the check of the
        server's certificate.[133X
  
        [33X[0;6YThe same effect is achieved for all [2XDownload[102X calls by setting the user
        preference [10XDownloadVerifyCertificate[110X (see [14X9.1-2[114X) to [9Xfalse[109X and omitting
        the [10XverifyCert[110X component from [3Xopt[103X.[133X
  
  [4X[32X  Example  [32X[104X
    [4X[25Xgap>[125X [27Xurl:= "https://www.gap-system.org/index.html";;[127X[104X
    [4X[25Xgap>[125X [27Xres1:= Download( url );;[127X[104X
    [4X[25Xgap>[125X [27Xres1.success;[127X[104X
    [4X[28Xtrue[128X[104X
    [4X[25Xgap>[125X [27XIsBound( res1.result ) and IsString( res1.result );[127X[104X
    [4X[28Xtrue[128X[104X
    [4X[25Xgap>[125X [27Xres2:= Download( Concatenation( url, "xxx" ) );;[127X[104X
    [4X[25Xgap>[125X [27Xres2.success;[127X[104X
    [4X[28Xfalse[128X[104X
    [4X[25Xgap>[125X [27XIsBound( res2.error ) and IsString( res2.error );[127X[104X
    [4X[28Xtrue[128X[104X
  [4X[32X[104X
  
  
  [1X9.1-2 [33X[0;0YUser preference [10XDownloadVerifyCertificate[110X[101X[1X[133X[101X
  
  [33X[0;0YThe  value [9Xtrue[109X (the default) means that the server's certificate is checked
  in  calls  of  [2XDownload[102X  ([14X9.1-1[114X),  such  that nothing gets downloaded if the
  certificate is invalid.[133X
  
  [33X[0;0YIf  the  value is [9Xfalse[109X then download methods are supposed to omit the check
  of  the  server's  certificate  (this  may  not be supported by all download
  methods).[133X
  
  [33X[0;0YOne  can  set  the  value  of the preference to be [10Xval[110X via [2XSetUserPreference[102X
  ([14XReference:   SetUserPreference[114X),  by  calling  [10XSetUserPreference(  "utils",
  "DownloadVerifyCertificate",  val  )[110X,  and  access  the  current  value  via
  [2XUserPreference[102X   ([14XReference:  UserPreference[114X),  by  calling  [10XUserPreference(
  "utils", "DownloadVerifyCertificate" )[110X.[133X
  
  [33X[0;0YWe recommend leaving this preference at its default value [9Xtrue[109X. Sometimes it
  can be necessary to change it, e.g. to work around issues with old operating
  systems  which  may  not  be  able  to correctly verify new certificates. In
  general  it is better to update such a system, but if that is not an option,
  then disabling certificate checks may be a good last resort.[133X
  
  
  [1X9.1-3 [33X[0;0YUser preference [10XDownloadMaxTime[110X[101X[1X[133X[101X
  
  [33X[0;0YThe  value [10X0[110X (the default) means that no timeout is set in calls of [2XDownload[102X
  ([14X9.1-1[114X).  If  the  value is a positive integer [22Xn[122X then those download methods
  that support a timeout will give up after [22Xn[122X seconds.[133X
  
  [33X[0;0YOne  can  set  the  value  of the preference to be [10Xval[110X via [2XSetUserPreference[102X
  ([14XReference:   SetUserPreference[114X),  by  calling  [10XSetUserPreference(  "utils",
  "DownloadMaxTime",  val  )[110X,  and access the current value via [2XUserPreference[102X
  ([14XReference:    UserPreference[114X),    by   calling   [10XUserPreference(   "utils",
  "DownloadMaxTime" )[110X.[133X
  
