Automated 'wget' patch downloads: issue resolution

My colleague, Don O'Malley, asked me to post the following on resolving issues using 'wget' to automate patch downloads.  'wget' is a popular download method, and is used by patch automation tools such as 'pca'.

Summary: You can use versions 1.10.x and 1.11.x of 'wget' but not version 1.11.  Details of options to use are set out below.  See also Patch Download Automation using wget.

SunSolve recently migrated to using Akamai for patch and patch cluster downloads, to provide customers with a faster and more reliable experience.

Some customers have experienced issues accessing patches using 'wget'.  Here's information on the issues and how to resolve them:

1) You must use a version of 'wget' which supports 'https'.

Why?

SunSolve's new patch download service is accessed by redirecting requests to https://getupdates2.sun.com, which subsequently redirects to https://a248.e.akamai.net (Akamai).
Which versions of 'wget' support 'https'?
'wget' version 1.10.x or later has 'https' support.
How can I check which version of 'wget' I am using?
Run the command 'wget --version'

2) You must use the '-O' or '--output-document' switch in 'wget' to provide an output filename.

Why?

The Akamai URI identifying a patch is very long.  By default 'wget' will name the downloaded file the same as the URI.  As the filename is too long an error is thrown and the download will fail.
Example of the correct syntax:
# /usr/sfw/bin/wget --http-user="xxxxxxxx" --http-passwd="xxxxxxx" --no-check-certificate "http://sunsolve.sun.com/pdownload.do?target=119255-01&method=h" -O /tmp/119255-01.zip

Example of some the output for a failing 'wget' request:

140778-01.zip?AuthParam=1251205908_479a27379ab5595128ae9170de4228c9&TUrl=L0QdUQV8Z4i0fdED3QTP3SJDWA8FMyaJsHfIWf4X29kTWQpKEzIbwqFuyRPZ&TicketId=3q3wk1CPNxhU&GroupName=SWUP&BHost=sdlc2h.sun.com&FilePath=%2Fpatches%2Fpatchroot%2Fall_unsigned%2F140778-01.zip&File=140778-01.zip: File name too long

Cannot write to `140778-01.zip? AuthParam=1251205908_479a27379ab5595128ae9170de4228c9&TUrl=L0QdUQV8Z4i0fdED3QTP3SJDWA8FMyaJsHfIWf4X29kTWQpKEzIbwqFuyRPZ&TicketId=3q3wk1CPNxhU&GroupName=SWUP&BHost=sdlc2h.sun.com&FilePath=%2Fpatches%2Fpatchroot%2Fall_unsigned%2F140778-01.zip&File=140778-01.zip' (Error 0).

3) If you are using 'wget' version 1.11.x you must use the '--auth-no-challenge' switch.

Why?

This is related to the manner in which 'wget' 1.11.x sends SunSolve a users Sun Online Account (SOA) information in this version of 'wget' (i.e. via '--http-user' & '--http-passwd'.)
Failure to include the '--auth-no-challenge' with 'wget' 1.11.x requests will result in the SunSolve Software License Agreement (SLA) being downloaded rather than the patch.
Example of the syntax for 'wget' 1.11.x users:
# /usr/sfw/bin/wget --auth-no-challenge --http-user="xxxxxxxx" --http-passwd="xxxxxxx" --no-check-certificate "http://sunsolve.sun.com/pdownload.do?target=119255-01&method=h" -O /tmp/119255-01.zip
Note, 'wget' version 1.11 does not have the '--auth-no-challenge' switch and so is not compatible with patch downloads from SunSolve.

4) You must provide 'wget' with direction on how to handle security certificate information.  Otherwise, patch downloads via 'wget' will fail.

Why?

Domains, getupdates2.sun.com & a248.e.akamai.net, are signed by trusted Certificate Authorities. (Verisign for Sun's and GTE Cybertrust for the case of Akamai.) Without a pointer to these certificates being provided to 'wget', download attempts will fail.
Which certs are required?
CN=GTE CyberTrust Global Root
CN=VeriSign Class 3 Secure Server CA - G2
What kind of error message can you expect to see from a failing 'wget' request?
ERROR: Certificate verification error for getupdates2.sun.com: self signed certificate in certificate chain
To connect to getupdates2.sun.com insecurely, use `--no-check-certificate'.
Unable to establish SSL connection.
Issue resolution:
If you wish to ignore this failure you can use the '--no-check-certificate' switch in 'wget'.  Example of the syntax:
# /usr/sfw/bin/wget --http-user="xxxxxxxx" --http-passwd="xxxxxxx" --no-check-certificate "http://sunsolve.sun.com/pdownload.do?target=119255-01&method=h" -O /tmp/119255-01.zip
If you wish to check against the certificates, you can use the '--ca-certificate' switch to point to a file containing the certificates.
http://sunsolve.sun.com/search/document.do?assetkey=1-9-240066-1 has an attachment called cacerts.pem, which is a concatenation of the two certificates.
If you save this file locally (eg to /tmp/cacerts.pem), you can use a syntax similar to:
# /usr/sfw/bin/wget --ca-certificate=/tmp/cacerts.pem --http-user="xxxxxxxx" --http-passwd="xxxxxxx" "http://sunsolve.sun.com/pdownload.pl?target=142284&method=h" -O /tmp/140778-01.zip

5) You may need to add firewall rules to enable 'wget' to work with SunSolve's new download service.

Why?

As the new download service is accessed by redirecting from http//:sunsolve.sun.com to https://getupdates2.sun.com initially and subsequently to https://a248.e.akamai.net, some customers may need to update their firewall rules to pass traffic from getupdates2.sun.com & a248.e.akamai.net in addition to sunsolve.sun.com.
How can I verify this?
Contact your System Administrator.

6) After associating a new contract to a SunSolve account there is a delay of up to 48 hours before 'wget' downloads will work for patches that the new contract should provide access to.

Additionally, customers registered in the Members Support Center must make an initial 'wget' call (which will fail) in order to trigger the synchronization process after associating a new contract to their party.

Why?

The delay is due to synchronization issues between SunSolve and the back-end access entitlement system.  Work is ongoing to reduce this delay.
What error message can you expect to see until this synchronization is complete ?
HTTP request sent, awaiting response... 403 You are not entitled to retrieve this content.

7) Attempts to download a patch README file by providing "method=r" in the URI is now failing.

Why?

Prior to the latest SunSolve release it was possible to download a patch's README file only via 'wget', using a syntax similar to :
# /usr/sfw/bin/wget --no-check-certificate --http-user="xxxxxxxx" --http-passwd="xxxxxxxx" "http://sunsolve.sun.com/pdownload.do?target=142284-01&method=r" -O /tmp/142284-01.README
There's a bug in the current SunSolve release this no longer works and attempts to download a patch README using this URI will result in a file of 0 Bytes being created.  This will be fixed at a later date.
Workaround:
Use "method=tr" to download a patch README file.  Example command syntax:
# /usr/sfw/bin/wget --no-check-certificate --http-user="xxxxxxxx" --http-passwd="xxxxxxxx" "http://sunsolve.sun.com/pdownload.do?target=142284-01&method=tr" -O /tmp/142284-01.README
Comments:

Let's do a thought experiment. If this tool was created by people who needed it to do their own daily work, what would it look like?

Posted by Mikael Gueck on August 26, 2009 at 06:10 PM IST #

What ever happenned to the KISS (Keep it simple, S..eriously) philosophy??.
Why waste CPU cycles on the server by encrypting files with https://?
And what the #### happened to good old FTP servers for crying out loud?

Sun is not alone in this behaviour.... IBM has copied you guys too. I came here after a Google search on that dreaded "self-signed certificate in chain" error message, after trying to download IBM's copycat office suite Lotus Symphony. Again, a convoluted process using https:// pages and authentication.

Control freaks have taken over the Net, I want to get off...

FC

Posted by Fernando Cassia on September 07, 2009 at 11:43 PM IST #

I'd be grateful for a correct syntax example for downloading recommended patch clusters. I'm having success with individual patches, but a request for 10_Recommended.zip fails. I have copied and rechecked syntax in a variety of wget examples and scripts.

Posted by Eugene Mosburg on September 22, 2009 at 12:46 PM IST #

Hi Eugene,

The following syntax will work to download Patch Clusters (I am downloading the Solaris 10 SPARC Recommended cluster in this example):

# /usr/sfw/bin/wget --no-check-certificate --http-user="SOA_USERNAME" --http-passwd="SOA_PASSWORD" "http://sunsolve.sun.com/pdownload.do?target=10_Recommended.zip" -O /tmp/10_Recommended.zip

There are a couple of points to note here:
\* SOA_USERNAME & SOA_PASSWORD are your Sun Online Account username/password that you use to log onto SunSolve.
\* All of the same requirements for wget, outlined in this blog (eg. https support and handling the certificate issues), apply to downloading patch clusters too.
\* This syntax will ignore checking security certificates. Details of how to perform security certificate checking are included in the blog entry above.
\* You need to have a support contract in order to be entitled to download patch clusters.
\* If you want to download other patch clusters but are unsure of the URI to provide wget in place of "http://sunsolve.sun.com/pdownload.do?target=10_Recommended.zip":
- Go to the Patch Cluster page on SunSolve (http://sunsolve.sun.com/show.do?target=patches/patch-access) and find the cluster you want to download (eg. Solaris 10 x86 Sun Alert Patch Cluster).
- Right click on the "Download" link under "Download Cluster" and select "Copy Link Location" (in Firefox anyway!)
- Paste the new link into the wget query in place of "http://sunsolve.sun.com/pdownload.do?target=10_Recommended.zip" (eg. "http://sunsolve.sun.com/pdownload.do?target=10_x86_SunAlert_Patch_Cluster.zip" for the Solaris 10 x86 Sun Alert Patch Cluster)

HTH,
-Don

Posted by Don O'Malley on September 23, 2009 at 07:33 AM IST #

Thanks, Don! Your example works fine. I discovered my error, had been appending "&method=h" to the URL (which DOES work for download of indvidual patches).

Posted by Eugene Mosburg on September 24, 2009 at 08:49 AM IST #

Folks,
Is there a way to download just the "patch_order" file i.e. included with Recommended Patch Clusters? (using wget)
TIA
- SV

Posted by SV on September 28, 2009 at 09:35 PM IST #

Hi SV!

Paraphrasing from Ed Clark, the cluster author:

The Cluster README contains the list of patches in the same order as the patch_order file, so you could potentially cut-and-paste from there.

Having said that, it's not identical to the patch_order file, as the patch_order file contains adjacent duplicate entries for certain broken SWUP (Software Update Manager) patches, which need to be applied twice to ensure they apply correctly. The README patch order does not contain these duplicate entries.

Best Wishes,

Gerry.

Posted by Gerry Haskins on September 29, 2009 at 05:26 AM IST #

Hi SV,

I'm afraid you can't download the patch_order file without downloading the whole cluster.

The only resources that are made available for download are the complete cluster and the cluster README file.

That said, the patch cluster README file will contain a section entitled "PATCHES INCLUDED" that will list the patches included in the cluster in the order in which they should be installed (ignoring the duplicate patch issue Gerry has noted previously).
The cluster README file also contains a date at the beginning of the file, so that you can use it to determine whether or not the patch cluster has been updated.

You can use wget to download the cluster README file directly by using a syntax similar to:
# wget --no-check-certificate --http-user="SOS_USERNAME" --http-passwd="SOA_PASSWORD" "http://sunsolve.sun.com/pdownload.do?target=10_Recommended.README" -O /tmp/10_Recommended.README

HTH,
-Don

Posted by Don O'Malley on September 29, 2009 at 05:54 AM IST #

I keep getting the terms and conditions page when I try to retrieve a readme file. How do I get past that page with wget?

-regards

Posted by Mike on October 19, 2009 at 12:58 PM IST #

Hi Mike,

Can you confirm the version of wget that you are using by running the 'wget --version' command please?

If the version is 1.11.x, then please add the '--auth-no-challenge' option to your wget command to workaround being sent the SunSolve SLA, as opposed to the patch download you are requesting.

HTH,
-Don

Posted by Don O'Malley on October 19, 2009 at 05:07 PM IST #

Nice blog entry. I thought Sun had changed the privileges for wget users with no support contracts as my wget had suddenly stopped working.

I noticed a typo in your blog entry. In the main blog entry you specify to use:

--no-auth-challenge

As you indicated to Mike on Oct 20, 2009, the correct option is --auth-no-challenge. My clue was wget telling me that --no-auth-challenge was not a wget option.

wget works great once again :)

Posted by Sean O'Neill on November 01, 2009 at 09:12 AM GMT #

Thanks Sean!

Typo corrected.

Best Wishes,

Gerry.

Posted by Gerry Haskins on November 01, 2009 at 07:57 PM GMT #

wget fails to download recommended cluster but works fine with single patch.
(sanitized output below)

Cannot get recommended cluster with wget

# wget --version
GNU Wget 1.11.4

# wget --auth-no-challenge --no-check-certificate --http-user="myuser" --http-passwd="mypassword" "http://sunsolve.sun.com/pdownload.do?target=10_Recommended.zip" -O file.zip
--2009-11-09 17:45:44-- http://sunsolve.sun.com/pdownload.do?target=10_Recommended.zip
Resolving sunsolve.sun.com... 192.18.108.40
Connecting to sunsolve.sun.com|192.18.108.40|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2009-11-09 17:45:47 ERROR 403: Forbidden.

However, getting a single patch works perfect

# wget --auth-no-challenge --no-check-certificate --http-user="myuser" --http-passwd="mypasswd" "http://sunsolve.sun.com/pdownload.pl?target=110380&method=h" -O file.zip
--2009-11-09 17:49:46-- http://sunsolve.sun.com/pdownload.pl?target=110380&method=h
Resolving sunsolve.sun.com... 192.18.108.40
Connecting to sunsolve.sun.com|192.18.108.40|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://sunsolve.sun.com/pdownload.do?target=110380&method=h [following]
--2009-11-09 17:49:46-- http://sunsolve.sun.com/pdownload.do?target=110380&method=h
Reusing existing connection to sunsolve.sun.com:80.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://getupdates2.sun.com/all_unsigned/110380-06.zip [following]
--2009-11-09 17:49:50-- https://getupdates2.sun.com/all_unsigned/110380-06.zip
Resolving getupdates2.sun.com... 198.232.168.157
Connecting to getupdates2.sun.com|198.232.168.157|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://a248.e.akamai.net/f/248/21808/15m/sun.download.akamai.com/21808/patches/patchroot/all_unsigned/110380-06.zip?AuthParam=1257807114_8d852b5eddc498ca0e61a039cfaa&TUrl=L0QdUQV8Z4i0fdED3QTP3FMyaJsHfIWf4X29kTWQpKEzIbwqFuyRPZ&TicketId=OxpU7H6lA==&GroupName=SWUP&BHost=sdlc3i.sun.com&FilePath=/patches/patchroot/all_unsigned/110380-06.zip&File=110380-06.zip [following]
--2009-11-09 17:49:54-- https://a248.e.akamai.net/f/248/21808/15m/sun.download.akamai.com/21808/patches/patchroot/all_unsigned/110380-06.zip?AuthParam=1257807114_8d852b0f745eddc40e61a039cfaa&TUrl=L0QdUQV8Z4i0fdED3QTP3SJDWA8FMyaJsHfIWf4X29KEzIbwqFuyRPZ&TicketId=OxpU7kH6lA==&GroupName=SWUP&BHost=sdlc3i.sun.com&FilePath=/patches/patchroot/all_unsigned/110380-06.zip&File=110380-06.zip
Resolving a248.e.akamai.net... 204.8.50.33, 204.8.50.64
Connecting to a248.e.akamai.net|204.8.50.33|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 327240 (320K) [application/zip]
Saving to: `file.zip'

100%[==================================================================================>] 327,240 --.-K/s in 0.07s

2009-11-09 17:49:54 (4.62 MB/s) - `file.zip' saved [327240/327240]

#

Posted by Asif on November 09, 2009 at 03:04 PM GMT #

Hi Asif,

The error message that you are seeing when attempting to download 10_Recommended.zip - i.e.:
HTTP request sent, awaiting response... 403 Forbidden
2009-11-09 17:45:47 ERROR 403: Forbidden.
usually indicated that you are not entitled to retrieve the resource that you are requesting.

This is usually a result of either:
- Not having a valid support contract.
- Not associating your support contract with your Sun Online Account.

You can verify the contracts associated with your account by logging on to SunSolve and clicking on "Update Account" (http://sunsolve.sun.com/edit-user-form.do) on the top right of the page.
You will see your contracts listed in the "Current Contracts" field.

BTW - I suspect the reason that you can download patch 110380-06 without any issue is because that patch does not require a support contract.

HTH,
-Don

Posted by Don O'Malley on November 10, 2009 at 04:39 AM GMT #

Hi,

I upgraded wget to the correct version, and my contract is still current (can login via SunSolve and download manually), and have no proxy in between, but I STILL keep getting the 403 Forbidden error. What might be the problem now? See below for my screen log please:

# ./downloadpatches.sh
Downloading unsigned patch 118815-06.
--2009-12-23 11:33:17-- http://sunsolve.sun.com/pdownload.pl?target=118815-06&method=h
Resolving sunsolve.sun.com... 192.18.108.40
Connecting to sunsolve.sun.com|192.18.108.40|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://sunsolve.sun.com/pdownload.do?target=118815-06&method=h [following]
--2009-12-23 11:33:18-- http://sunsolve.sun.com/pdownload.do?target=118815-06&method=h
Reusing existing connection to sunsolve.sun.com:80.
HTTP request sent, awaiting response... 403 Forbidden
2009-12-23 11:33:18 ERROR 403: Forbidden.

Posted by Wim Olivier on December 23, 2009 at 01:47 AM GMT #

Hi Wim,

Apologies for the delay in coming back to you on this.

I'm assuming that the downloadpatches.sh script you are using was generated by the new PatchFinder tool, right?

There was a later version of wget (1.12) delivered in patches 125215-03 & 125216-03, which were released at the beginning of December.
The downloadpatches.sh script does not yet handle wget version 1.12, as it was not available at the time the script was written.

As a result, the --auth-no-challenge option is not set for wget version 1.12, but it needs to be.

I am working on getting the script that is downloaded via PatchFinder updated, but in the meantime if you wish to manually edit the downloadpatches.sh script to replace the lines:

echo $VERSION | grep "1.11." > /dev/null
if [ $? = 0 ]; then

with

if [ $SUB_VERSION -ge "11" ]; then

the script will work.

If you are continuing to experience difficulties, please let me know.

Best,
-Don

Posted by Don O'Malley on January 04, 2010 at 04:08 AM GMT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

This blog is to inform customers about patching best practice, feature enhancements, and key issues. The views expressed on this blog are my own and do not necessarily reflect the views of Oracle. The Documents contained within this site may include statements about Oracle's product development plans. Many factors can materially affect these plans and the nature and timing of future product releases. Accordingly, this Information is provided to you solely for information only, is not a commitment to deliver any material code, or functionality, and SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. The development, release, and timing of any features or functionality described remains at the sole discretion of Oracle. THIS INFORMATION MAY NOT BE INCORPORATED INTO ANY CONTRACTUAL AGREEMENT WITH ORACLE OR ITS SUBSIDIARIES OR AFFILIATES. ORACLE SPECIFICALLY DISCLAIMS ANY LIABILITY WITH RESPECT TO THIS INFORMATION. ~~~~~~~~~~~~ Gerry Haskins, Director, Software Lifecycle Engineer

Search

Categories
Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today