wdas / reposado Goto Github PK

Host Apple Software Updates on the hardware and OS of your choice.

License: Other

Python 100.00%

reposado's Introduction

macOS Big Sur important information
In macOS Big Sur, Apple has removed the ability for softwareupdate to be pointed to a non-Apple sucatalog. This means you cannot use a Reposado server to serve Apple software updates to Big Sur (and presumably later versions of macOS) clients.

INTRODUCTION

Reposado is a set of tools written in Python that replicate the key functionality of Mac OS X Server's Software Update Service.

LICENSE

Reposado is licensed under the new BSD license.

DISCUSSION GROUP

Discussion for users and developers of Reposado is here.

FEATURES AND CAPABILITIES

Reposado, together with Python, the "curl" binary tool and a web server such as Apache 2, enables you to host a local Apple Software Update Server on any hardware and OS of your choice.

Reposado contains a tool (repo_sync) to download Software Update catalogs and (optionally) update packages from Apple's servers, enabling you to host them from a local web server.

Additionally, Reposado provides a command-line tool (repoutil) that enables you to create any arbitrary number of "branches" of the Apple catalogs. These branches can contain any subset of the available updates. For example, one could create "testing" and "release" branches, and then set some clients to use the "testing" branch catalog to test newly-released updates. You would set most of your clients to use the "release" branch catalog, which would contain updates that had been through the testing process.

If you configure Reposado to also download the actual updates as well as the catalogs, you can continue to offer updates that have been superseded by more recent updates. For example, if you are currently offering the 10.6.7 updates to your clients, and Apple releases a 10.6.8 update, you can continue to offer the (deprecated) 10.6.7 update until you are ready to release the newer update to your clients. You can even offer the 10.6.7 update to your "release" clients while offering the 10.6.8 update to your "testing" clients. Offering "deprecated" Apple Software Updates is a feature that is difficult with Apple's tools.

LIMITATIONS AND DEPENDENCIES

Apple's Software Update Service does a few things. Primarily, it replicates software updates from Apple's servers, downloading them to a local machine. Secondly, it functions as a web server to actually serve these updates to client machines. Reposado does not duplicate the web server portion of Apple's Software Update Service. Instead you may use any existing web server you wish.

Reposado also currently relies on the command-line "curl" binary to download updates from Apple's servers. curl is available on OS X, RedHat Linux, and many other OSes, including Win32 and Win64 versions. See http://curl.haxx.se for more information.

MORE INFO

More information and basic documentation is available here: https://github.com/wdas/reposado/tree/master/docs

reposado's People

Contributors

Stargazers

Watchers

Forkers

brentbb robrepp sashka mattsmacblog klodr timsutton dsqmoore guniorobot vandehey natewalck hjuutilainen tedwardd jctudela unixorn bennettp123 crazyleb13 asurasunil oneiroi joraff kenergy amedeomantica tonyliu2ca celacela10 joshuagarr jessepeterson elflames vtvincent drucko smkudelko shenangovalleyavclub newfuturestech mogh chymb ejc153 uikit0 saihaynes ckindley bsdhelper dannooooo mikep345678 mschwieger carsomyr alex-osx karnlund djbdjb00djb harrisony raqibhakim jschripsema mwohlgemuth mlg415 heathseals benbasscom opsemanthano niccol wilson333 bryonwausau alexandregz geoff-nixon digims luser cserfoss michalmmac jnordlund cvgs magervalp dinkin lonit-was apettinen ianlauin yunzheng tindoge iserko capricorntb galaxysd nickkarras waisanen tatoalves weswhet alexxnica kryndex hhandal jamyn cncharles18 sphen13 tkuennen w0de kevinstrick cloudspare neilmartin83 chefaustin peterhole jacobfgrant brinnjoyce gmarnin epackorigan bikram990 connectionmaster hustjs nathanperkins xiaogui2005

reposado's Issues

Enhancement Request

Instead of the --configure option being interactive can repoutil be modified so it can be setup using a one liner? It would be nice to be able to just pass the two paths and the URL to get initial config setup.

errors when using repo_sync

Hi,

While trying to run repo_sync on a Mac running 10.5.8 server, I get the following messages that I didn't receive previously:

450 products found in http://swscan.apple.com/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog
Traceback (most recent call last):
File "/usr/local/bin/repo_sync", line 674, in
main()
File "/usr/local/bin/repo_sync", line 670, in main
download_packages=download_packages)
File "/usr/local/bin/repo_sync", line 637, in sync
reposadocommon.writeProductInfo(products)
File "/usr/local/bin/reposadolib/reposadocommon.py", line 266, in writeProductInfo
writeDataToPlist(product_info_dict, 'ProductInfo.plist')
File "/usr/local/bin/reposadolib/reposadocommon.py", line 223, in writeDataToPlist
os.path.join(metadata_dir, filename))
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/plistlib.py", line 93, in writePlist
writer.writeValue(rootObject)
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/plistlib.py", line 248, in writeValue
self.writeDict(value)
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/plistlib.py", line 277, in writeDict
self.writeValue(value)
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/plistlib.py", line 248, in writeValue
self.writeDict(value)
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/plistlib.py", line 277, in writeDict
self.writeValue(value)
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/plistlib.py", line 256, in writeValue
raise TypeError("unsuported type: %s" % type(value))
TypeError: unsuported type: <type 'long'>

Any help as to the cause and solution would be helpful and appreciated.

Thanks,
Adam

Repo_sync - HTTP result 200: Connection established

Since a few days I get this when I run the repo_sync:

repo_sync run started
Could not replicate https://swscan.apple.com/content/catalogs/others/index-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog: HTTP result 200: Connection established
Could not replicate https://swscan.apple.com/content/catalogs/others/index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog: HTTP result 200: Connection established
repo_sync run ended

I don't know why. Because I changed nothing.
What means the HTTP result 200?

Please can you help me?

How can I update complete OS versions via Reposado

Hi,
I have just setup a reposado server with Margarita on a Debian VM, everything looks quite good so far.
I have the 10.11, 10.12 and 10.13 Catalogs in my preferences.plist and it finds about 500-550 products for each of them so it all seems to be working
However I can't find the "Install macOS High Sierra" update for 10.12 in Margarita.
I have one for 10.11 which has a D behind it and is not listed in Apples branch.
I assume the D stands for deprecated?
Do I have to add a different catalog for full OS version updates or is there anything else I can try?

Thanks

dan

Repo_sync - HTTP result 200: Connection established - again

I am seeing this as well with the latest repo_sync. We do have microsoft's TMG proxying by a parent entity. I have no access to it or insight into its configuration. Suggestions?

Setup issue

I've downloaded the pIistlib.py, but when I execute this : ./repoutil --configure
I get this error :

Traceback (most recent call last):
File "./repoutil", line 43, in
from reposadolib import reposadocommon
File ".../wdas-reposado-692f075/code/reposadolib/reposadocommon.py", line 44, in
import plistlib
File ".../wdas-reposado-692f075/code/plistlib.py", line 143
self.file.write(b'\n')
^
SyntaxError: invalid syntax

Please help

repoutil does not catch errors for nonexistent catalogs

I did a git pull on the repository today and then immediately began attempting to add products to my stable branch. This caused repoutil to throw the following exception because the updated repo_sync was not run beforehand, therefore there wasn't a catalog for Mavericks.

Traceback (most recent call last):
File "/usr/local/reposado/code/repoutil", line 687, in
main()
File "/usr/local/reposado/code/repoutil", line 675, in main
add_product_to_branch(params)
File "/usr/local/reposado/code/repoutil", line 390, in add_product_to_branch
reposadocommon.writeAllBranchCatalogs()
File "/usr/local/reposado/code/reposadolib/reposadocommon.py", line 352, in writeAllBranchCatalogs
writeBranchCatalogs(localcatalogpath)
File "/usr/local/reposado/code/reposadolib/reposadocommon.py", line 357, in writeBranchCatalogs
catalog = plistlib.readPlist(localcatalogpath)
File "/usr/lib/python2.7/plistlib.py", line 75, in readPlist
pathOrFile = open(pathOrFile)
IOError: [Errno 2] No such file or directory: '/var/www/reposado/html/content/catalogs/others/index-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog'

Simply running repo_sync as you suggested in IRC was enough to resolve the issue. It would be nice if repoutil could have caught the error and done that for me.

./repoutil --deprecated

Running

./repoutil --deprecated
on macOS 10.14 with Python 2.7.10 lead to

Traceback (most recent call last):
  File "./repoutil", line 743, in <module>
    main()
  File "./repoutil", line 701, in main
    list_deprecated(sort_order=options.sort, reverse_sort=options.reverse)
  File "./repoutil", line 271, in list_deprecated
    list_products(sort_order, reverse_sort, list_of_productids)
  File "./repoutil", line 327, in list_products
    product_list.sort(key=lambda item: item['sort_key'], reverse=reverse_sort)
TypeError: an integer is required

Does anyone have an idea why ?

Reposado Probem with 10.9 Clients

Hello :-)

I have some Problem with my configuration of my reposado server.
I cannot connect the clients to the update server.
The following error occurs when I try to start the softwareupdate:

Finding availbale software
Die Daten können nicht vom Softwareaktualisierungsserver (hostname.yourorg.de) geladen werden.

in english -> "An error has occurred. Can't load data from the Software Update Server" (Server.domain.org).

I give the clients the following conf:

defaults write /Library/Preferences/com.apple.SoftwareUpdate CatalogURL http://"myhostname.myorg.de:8089"/content/catalogs/others/index-10.9-mountainlion-lion-snowleopard-leopard.merged-1_"my-branch"_sucatalog

I also trying it with Port 80. Result: URL not know/allowed...
I use Cent OS with the following configuration:

http://www.adminsys.ch/2012/09/23/install-reposado-margarita-centos-red-hat-enterprise-linux/

What do I wrong? Hope you can give me a hint.

Greetings

xfiles

docs tweak

It's all @mwohlgemuth's fault, but the last paragraph doesn't specifically mention that the option you'd call is --purge-product(s), it currently sounds like --remove-product will also apply in this context.

Reposado: BridgeOSUpdateCustomer & macOS Catalina Security Update 2021-002

Hi all

I have an issue with an update on the reposado and an installation of an update on a client.

BridgeOSUpdateCustomer
There are 2 updates of BridgeOSUpdateCustomer, 071-29167::1 and 071-29167. The 071-29167 can be selected and assigned to a catalog. for the update 071-29167::1, the same steps are not working. Already removed the update from Productinfo.plist and synced freshly, but the error is the same: Catalog branch doesn't exist!

macOS Catalina Security Update 2021-002
I can't install this security update on the catalina clients. I think, BridgeOSUpdateCustomer (071-29167::1) is needed for this because when I remove Update server profile from the client, the security update can be downloaded and installed. I get the error:

Download failed
An error occurred while downloading the selected updates etc.

any ideas?

Kind Regards
Fuat

OS X Sierra URL Not Supported

HI,

I configured all on my OS X Server.
I have a problem with the client running OS X Sierra.

When I try to check for update, I have "URL not supported" error.

Any suggestion?

I pointed my client to:

sudo defaults write /Library/Preferences/com.apple.SoftwareUpdate CatalogURL http://su.myserver.com:80/index.sucatalog

with .htaccess file

repoutil --updates: Columns not wide enough

In the output of the command "repoutil --updates --sort date", the first two columns (id/key and title) are not wide enough to hold some of the products.

Examples:

Key/ID fits in column

041-4535 Xcode Update 3.2.6 2012-03-20 ['MacAll', 'MacPilot', 'MacProduction']

Key/ID is too large for column

11B26_ServerEssentials Server Essentials 1.0 2012-03-20 ['MacAll']

Title fits in column

031-1228 Mavericks Developer Preview Recovery Update 1.0 2013-10-22 []

Title is too large for column

031-1048 MacBook Pro (Retina, 13-inch, Late 2013) Software Update 1.0 2013-10-22 []

I tried editing the source myself, but I've never used python and am clearly doing something wrong. Please fix!

Unable to configure Mac OS X 11 Big Sur correctly

We are using Reposado for quite a while now and since the switch to 11.0 our clients can no longer talk to the caching server. I managed to add the catalog and I can retrieve the content via curl, however, softwareupdate always returns:
The operation couldn’t be completed. (NSURLErrorDomain error -1012.)

I tried to change the repository URL in the preferences file to use HTTPS, to no avail. The catalog looks correct though.

Does anyone use this setup already?

Issue while running repo_sync

Hi, we've been using reposado for a while now, but when I went to sync today it threw out the following error...:

Am I missing something here or have I done something wrong?

Cheers.

Some content does not get downloaded

I've tried repo_sync several times, and there is some file that don't get downloaded.

$ repo_sync
repo_sync run started
283 products found in http://swscan.apple.com/content/catalogs/index.sucatalog
Building index.sucatalog...
186 products found in http://swscan.apple.com/content/catalogs/index-1.sucatalog
Building index-1.sucatalog...
283 products found in http://swscan.apple.com/content/catalogs/others/index-leopard.merged-1.sucatalog
Building index-leopard.merged-1.sucatalog...
360 products found in http://swscan.apple.com/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog
Building index-leopard-snowleopard.merged-1.sucatalog...
586 products found in http://swscan.apple.com/content/catalogs/others/index-lion-snowleopard-leopard.merged-1.sucatalog
Could not replicate http://swcdn.apple.com/content/downloads/11/05/041-0925/g27es04pw9re5ggrfp3vuf8ew6r53asfz8/041-0925.English.dist: HTTP result 200: OK
Error reading /reposado/data/content/downloads/11/05/041-0925/g27es04pw9re5ggrfp3vuf8ew6r53asfz8/041-0925.English.dist: [Errno 2] No such file or directory: '/reposado/data/content/downloads/11/05/041-0925/g27es04pw9re5ggrfp3vuf8ew6r53asfz8/041-0925.English.dist'
Could not get data from dist file: /reposado/data/content/downloads/11/05/041-0925/g27es04pw9re5ggrfp3vuf8ew6r53asfz8/041-0925.English.dist
Building index-lion-snowleopard-leopard.merged-1.sucatalog...
WARNING: did not add product 041-0925 to catalog index-lion-snowleopard-leopard.merged-1.sucatalog.apple because it has not been downloaded.
358 products found in http://swscan.apple.com/content/catalogs/others/index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Could not replicate http://swcdn.apple.com/content/downloads/11/05/041-0925/g27es04pw9re5ggrfp3vuf8ew6r53asfz8/041-0925.English.dist: HTTP result 200: OK
Error reading /reposado/data/content/downloads/11/05/041-0925/g27es04pw9re5ggrfp3vuf8ew6r53asfz8/041-0925.English.dist: [Errno 2] No such file or directory: '/reposado/data/content/downloads/11/05/041-0925/g27es04pw9re5ggrfp3vuf8ew6r53asfz8/041-0925.English.dist'
Could not get data from dist file: /reposado/data/content/downloads/11/05/041-0925/g27es04pw9re5ggrfp3vuf8ew6r53asfz8/041-0925.English.dist
Building index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog...
WARNING: did not add product 041-0925 to catalog index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.apple because it has not been downloaded.
repo_sync run ended

Issue with adding High Sierra

Team

I am trying to update reposado to include updates for High Sierra, the issue I am running into is the new catalog for High Sierra does not appear to create, I edited the .htaccess file and my client can see that change judging from the messages I am getting back

bash-3.2# curl --user-agent "Darwin/17" http://mydomain.com:8088/index.sucatalog > /tmp/release
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 324 100 324 0 0 26470 0 --:--:-- --:--:-- --:--:-- 27000
bash-3.2# tail -5 /tmp/release

<title>404 Not Found</title>

Not Found

The requested URL /content/catalogs/others/index-10.13-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1_production.sucatalog was not found on this server.

I have checked the settings on the webserver and they seem to be correct and I've tried a manual repo sync but that doesn't see the catalog either. If you could tell me what glaringly obvious thing I am missing I would be very grateful.

Regards

Alex

Whitespace easily makes its way into prefs via drag/drop, quick fix

I set and semi-messed this up for the first time while running repoutil --config due to raw_input leaving whitespace at the end of the path, since I drag/dropped into the terminal window to set both doc paths... which in turn messed with my URL. On line 110 of reposadocommon.py could we add a conditional, something like
if newvalue[-1] = ' ':
newvalue = newvalue[:-1]
just to prevent noob mistakes and ever-so-slightly sanitize the input? Thanks for your consideration.

The operation couldn’t be completed. (NSURLErrorDomain error -1100.)

I'm not sure if I'm missing a step. But I've setup Reposado and Margarita on macOS 10.13.6. I can see 1788 updates listed in Margarita, and the repo_sync finishes successfully. I'll run this command to point to my software update catalog:

$ sudo softwareupdate --set-catalog https://<SERVER.DOMAIN>/reposado/html/content/catalogs/others/index-10.13-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1_testing.sucatalog

I can even browse that in safari and it shows the XML.

But when I try to list updates I get an error:
$ softwareupdate --list --verbose
Software Update Tool

Finding available software
The operation couldn’t be completed. (NSURLErrorDomain error -1100.)

I see the same error if I use the AppStore to look for updates. It had been working when I had used the Apple SUS on Sierra. Am I missing something?

Human readable file sizes

Would you consider a quick change or a patch for some more human readable output?

I'd like to see this:

Downloading 1126391 bytes from http://swscan.apple.com/content/catalogs/index.sucatalog...
301 products found in http://swscan.apple.com/content/catalogs/index.sucatalog
Downloading 14818 bytes from http://swcdn.apple.com/content/downloads/20/44/041-   7449/w8gzvftbg47l4ozr8fhm8h75073wlb1t4u/Aperture3.4.1Update.smd...
Downloading 578113722 bytes from http://swcdn.apple.com/content/downloads/20/44/041-7449/w8gzvftbg47l4ozr8fhm8h75073wlb1t4u/Aperture3.4.1Update.pkg.

Turned into this:

Downloading 1 MB from http://swscan.apple.com/content/catalogs/index.sucatalog...
301 products found in http://swscan.apple.com/content/catalogs/index.sucatalog
Downloading 1.4 kB from http://swcdn.apple.com/content/downloads/20/44/041-7449/w8gzvftbg47l4ozr8fhm8h75073wlb1t4u/Aperture3.4.1Update.smd...
Downloading 551 MB from http://swcdn.apple.com/content/downloads/20/44/041-7449/w8gzvftbg47l4ozr8fhm8h75073wlb1t4u/Aperture3.4.1Update.pkg.

Change rewrite conditions for OSX 10.11 in nginx.conf

Good morning,

i verified tomorrow that El Capitain clients do ask for catalogs.sucatalog instead of index(.*).sucatalog, so i think that the correct rewrite should be:

if ( $http_user_agent ~ "Darwin/15" ){
rewrite ^/index(.*).sucatalog$ /content/catalogs/others/index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1$1.sucatalog last;
}

and not the one reported in docs/URL_rewrites.md file.

Problem running on Ras Pi

I installed this on a raspberry pi, when I run setup.py I get an error:

root@xxxxxxxxx:/etc/reposado# sudo ./setup.py
./setup.py: 1: ./setup.py: import: not found
./setup.py: 2: ./setup.py: import: not found
from: can't read /var/mail/distutils.core
./setup.py: 12: ./setup.py: Syntax error: "(" unexpected (expecting "then")

Client not downloading .pkg files/It may have been corrupted or tampered with during downloading

Hello,

We are using reposado (ubuntu/nginx) to host updates to our OS X clients (10.10.3). I now have a problem when i try to install updates on a clients.
When i try to install with managesofwateupdate, i get the following error message:

Installing iTunes...
0 Error downloading iTunes: “iTunes” could not be verified. It may have been corrupted or tampered with during downloading. Click Update to download and check the update again.
Done.
Error downloading updates.
WARNING: Apple update iTunes, zzzz031-31871 failed to install. No record of success or failure.

I checked download location (Library/Managed Installs/swupd/content/downloads/34/21/zzzz031-31871/oiafge5g10rnspc2kdm3qusk1dky3jbhut) on client and i noticed that all .pkg files are missing.

When i check my pilot branch catalog file from server, i can see that all files are listed. I can also download all missing pkg files with curl on client.

I checked nginx access log from server and noticed that all pkg files (+1 .dist file) are getting "Software%20Update (unknown version) CFNetwork/720.3.13 Darwin/14.3.0 (x86_64)".

10.0.72.77 - - [20/Aug/2015:09:34:29 +0300] "GET /index_pilot.sucatalog HTTP/1.1" 200 8877 "-" "managedsoftwareupdate/2.3.0.2519 Darwin/14.3.0 (x86_64) (iMac8,1)"
10.0.72.77 - - [20/Aug/2015:09:34:29 +0300] "GET /content/downloads/34/21/zzzz031-31871/oiafge5g10rnspc2kdm3qusk1dky3jbhut/031-31871.English.dist HTTP/1.1" 200 6644 "-" "Software%20Update (unknown version) CFNetwork/720.3.13 Darwin/14.3.0 (x86_64)"
10.0.72.77 - - [20/Aug/2015:09:34:30 +0300] "GET /content/downloads/34/21/zzzz031-31871/oiafge5g10rnspc2kdm3qusk1dky3jbhut/CoreADI.pkg HTTP/1.1" 200 1895738 "-" "Software%20Update (unknown version) CFNetwork/720.3.13 Darwin/14.3.0 (x86_64)"
10.0.72.77 - - [20/Aug/2015:09:34:30 +0300] "GET /content/downloads/34/21/zzzz031-31871/oiafge5g10rnspc2kdm3qusk1dky3jbhut/iTunesAccess.pkg HTTP/1.1" 200 687421 "-" "Software%20Update (unknown version) CFNetwork/720.3.13 Darwin/14.3.0 (x86_64)"
10.0.72.77 - - [20/Aug/2015:09:34:31 +0300] "GET /content/downloads/34/21/zzzz031-31871/oiafge5g10rnspc2kdm3qusk1dky3jbhut/MobileDevice.pkg HTTP/1.1" 200 32701834 "-" "Software%20Update (unknown version) CFNetwork/720.3.13 Darwin/14.3.0 (x86_64)"
10.0.72.77 - - [20/Aug/2015:09:34:31 +0300] "GET /content/downloads/34/21/zzzz031-31871/oiafge5g10rnspc2kdm3qusk1dky3jbhut/CoreFP.pkg HTTP/1.1" 200 26885432 "-" "Software%20Update (unknown version) CFNetwork/720.3.13 Darwin/14.3.0 (x86_64)"
10.0.72.77 - - [20/Aug/2015:09:34:34 +0300] "GET /content/downloads/34/21/zzzz031-31871/oiafge5g10rnspc2kdm3qusk1dky3jbhut/iTunesX.pkg HTTP/1.1" 200 164633993 "-" "Software%20Update (unknown version) CFNetwork/720.3.13 Darwin/14.3.0 (x86_64)"
1

These files were downloded ok.
10.0.72.77 - - [20/Aug/2015:09:51:53 +0300] "GET /content/downloads/34/21/zzzz031-31871/oiafge5g10rnspc2kdm3qusk1dky3jbhut/iTunesX.smd HTTP/1.1" 200 98472 "-" "managedsoftwareupdate/2.3.0.2519 Darwin/14.3.0 (x86_64) (iMac8,1)"
10.0.72.77 - - [20/Aug/2015:09:51:53 +0300] "GET /content/downloads/34/21/zzzz031-31871/oiafge5g10rnspc2kdm3qusk1dky3jbhut/CoreADI.pkm HTTP/1.1" 200 599 "-" "managedsoftwareupdate/2.3.0.2519 Darwin/14.3.0 (x86_64) (iMac8,1)"
10.0.72.77 - - [20/Aug/2015:09:51:53 +0300] "GET /content/downloads/34/21/zzzz031-31871/oiafge5g10rnspc2kdm3qusk1dky3jbhut/iTunesX.pkm HTTP/1.1" 200 2167 "-" "managedsoftwareupdate/2.3.0.2519 Darwin/14.3.0 (x86_64) (iMac8,1)"
1

Do you guys know what could cause this?

FeatureRequest: MultiLingualVoice

i'd like to have an option to skipt downloading all but some of the "MultiLigualVoice_xx_xx" and "MLVUpdate_xx_xx" files.

I'm downloading since 14 hours and i'm far from finishing !!

looking at my repo, the files actually to handle are

MultiLingualVoice_cc_CC_name
MLV_cc_CC_name
MLVUpdate_cc_CC_name
SpeechRecognitionCoreLanguage_cc_CC

something like

<key>OnlyVoices</key>
<array>
  <string>de_DE</string>
  <string>en_US</string>
</array>

Unknown cipher in list

Have some problem with sync.
Reposado output:

[root@share code]# ./repo_sync
repo_sync run started
280 products found in http://swscan.apple.com/content/catalogs/index.sucatalog
Building index.sucatalog...
Building index_osx.sucatalog...
186 products found in http://swscan.apple.com/content/catalogs/index-1.sucatalog
Building index-1.sucatalog...
Building index-1_osx.sucatalog...
280 products found in http://swscan.apple.com/content/catalogs/others/index-leopard.merged-1.sucatalog
Building index-leopard.merged-1.sucatalog...
Building index-leopard.merged-1_osx.sucatalog...
359 products found in http://swscan.apple.com/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog
Building index-leopard-snowleopard.merged-1.sucatalog...
Building index-leopard-snowleopard.merged-1_osx.sucatalog...
585 products found in http://swscan.apple.com/content/catalogs/others/index-lion-snowleopard-leopard.merged-1.sucatalog
Could not replicate http://swcdn.apple.com/content/downloads/10/21/031-12109/o0mghmdsr9638ac0ydmnxqnlzkkqaxsvi1/MacBookAirMid2012SoftwareUpd1.0.dst/031-12109.sk.dist: HTTP result 200: OK
Building index-lion-snowleopard-leopard.merged-1.sucatalog...
Building index-lion-snowleopard-leopard.merged-1_osx.sucatalog...
427 products found in http://swscan.apple.com/content/catalogs/others/index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Building index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog...
Building index-mountainlion-lion-snowleopard-leopard.merged-1_osx.sucatalog...
Could not replicate https://swscan.apple.com/content/catalogs/others/index-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog: Error 59: Unknown cipher in list: HIGH
Could not replicate https://swscan.apple.com/content/catalogs/others/index-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog: Error 59: Unknown cipher in list: HIGH
Could not replicate https://swscan.apple.com/content/catalogs/others/index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog: Error 59: Unknown cipher in list: HIGH
repo_sync run ended

But links seems to be working:

wget https://swscan.apple.com/content/catalogs/others/index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
--2015-10-11 11:35:01-- https://swscan.apple.com/content/catalogs/others/index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Распознаётся swscan.apple.com (swscan.apple.com)… 17.164.1.22
Подключение к swscan.apple.com (swscan.apple.com)|17.164.1.22|:443... соединение установлено.
HTTP-запрос отправлен. Ожидание ответа... 200 OK
Длина: 4325608 (4,1M) [application/x-apple-plist]
Сохранение в: «index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.2»

index-10.11-10.10-10.9-mountainlion-lion-snowleopard 100%[=======================================================================================================================>] 4,12M 167KB/s за 29s

2015-10-11 11:35:31 (146 KB/s) - «index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.2» сохранён [4325608/4325608]

tail -n5 index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
45a9dc67381d81b4af460c703b0dded9d4a3a96c
MetadataURL
https://swdist.apple.com/content/downloads/25/13/031-17334/o1b83gtjckvj2yvhdx1v3sz0nrpta28tjv/RicePaper-FoldedCard_E6C737BA-9C4A-4693-82F7-A3EE22555C4D.pkm
URL

repo_sync stops with error since Oct.24.

this is the output i get:

Traceback (most recent call last): File "/usr/local/bin/repo_sync", line 852, in <module> main() File "/usr/local/bin/repo_sync", line 848, in main download_packages=download_packages) File "/usr/local/bin/repo_sync", line 663, in sync 'Could not replicate %s: %s', catalog_url, err) File "/usr/local/bin/reposadolib/reposadocommon.py", line 235, in print_stderr output = concat_message(msg, *args) File "/usr/local/bin/reposadolib/reposadocommon.py", line 192, in concat_message args = [str_to_ascii(arg) for arg in args] File "/usr/local/bin/reposadolib/reposadocommon.py", line 185, in str_to_ascii return s.decode('ascii', 'ignore') AttributeError: 'ReplicationError' object has no attribute 'decode'

any advice what to do about it?
running on Ubuntu 18.04.2.

RFE: repoutil --add argument order problem

The order of arguments to repoutil --add et al. is a little unlucky with BRANCH_NAME coming last. If it was the first argument, one could easily use xargs to add a bunch of products to a branch in one go.
Surely it would break any script out there to just reverse the logic, but maybe it could be added as an alternative along the lines of the -t argument to GNU cp, mv etc.
Use case: I have a scriptlet that parses the catalog and prints the ids of all products for 10.1[34] so I can --add them for the chained-reposado-pattern to only download what's relevant on our site.
Workarounds abound, yet it would be one of those little things that make life a little more comfy.

Running repo_sync gives me errors recently

Hello, recently I notice that running a repo_sync gives me:

repo_sync run started
Traceback (most recent call last):
File "/usr/local/asus/reposado/code/repo_sync", line 852, in
main()
File "/usr/local/asus/reposado/code/repo_sync", line 848, in main
download_packages=download_packages)
File "/usr/local/asus/reposado/code/repo_sync", line 646, in sync
products = reposadocommon.getProductInfo()
File "/usr/local/asus/reposado/code/reposadolib/reposadocommon.py", line 311, in getProductInfo
return getDataFromPlist('ProductInfo.plist')
File "/usr/local/asus/reposado/code/reposadolib/reposadocommon.py", line 284, in getDataFromPlist
os.path.join(metadata_dir, filename))
File "/usr/lib/python2.7/plistlib.py", line 78, in readPlist
rootObject = p.parse(pathOrFile)
File "/usr/lib/python2.7/plistlib.py", line 406, in parse
parser.ParseFile(fileobj)
File "/usr/lib/python2.7/plistlib.py", line 418, in handleEndElement
handler()
File "/usr/lib/python2.7/plistlib.py", line 470, in end_string
self.addObject(self.getData())
File "/usr/lib/python2.7/plistlib.py", line 425, in addObject
self.stack[-1][self.currentKey] = value
TypeError: list indices must be integers, not str

Not sure why, I've updated to latest source but did not resolve.

Running on Ubuntu 16.04 LTS

Feature request

Not sure where else to put this. When tailing the log, the "Downloading etc etc etc" message doesn't appear until after the download has occurred. So in the case of large files that may take multiple minutes to download, one has no idea if anything is actually happening without opening a second utility like iotop. If possible it would be nice to have the message written to the log before the even occurs.

example, below took ~10 minutes to display because it didn't show until after the download happened:
Feb 01 10:42:42 Downloading 438188881 bytes from http://swcdn.apple.com/content/downloads/05/34/031-12187/t614kl532jwk6pqkri5uzvmv3rmsnfcb3r/BootCampESD.pkg...

Exclude repositories from repo_sync

Hello Greg,

How are you? I have a question please.
How can I delete repositories and exclude them from the repo_sync?
I do not need updates for mountainlion and older. So I can save diskspace.
Is it possible to delete only the catalogs I do not need? The problem is I cannot
see which Update is for what OS in the download folder.
And where I have to exclude the repositories in the configuration?
Only in the preferences.plist?

Thank you!

Reposado user cannot use crontab

Am i just dumb or what's the matter when...

% sudo puppet agent -t
... 
Notice: /Stage[main]/Reposado/Cron[repo_sync]/ensure: created
The user reposado cannot use this program (crontab)
Notice: Applied catalog in 5.94 seconds

If i sudo su - reposado and crontab -e, on the first time, i'm prompted for which editor to use, but that doesn't change anything.

Possibly related to that, i don't see a git checkout of the reposado repo (main.pp line 68, vcsrepo -- i wish i knew how to create a direct link from this issue to the relevant line in the code), so in effect, this module doesn't do anything for me :)

Enhancement: Option to automatically remove config-data on downloaded updates

I'm excited for the munki change allowing unattended Apple updates discussed here https://groups.google.com/d/msg/munki-dev/o6WtjFOL74k/kT36MAOPDgAJ

In the current workflow removing the config-data is tedious as the admin needs to go thru updates and understand which updates have config-data. Some are obvious, like XProtect and Gatekeeper, but some sneak up, like the recent MRT configuration update.

The enhancement I'm proposing is an option/preference that would allow all newly downloaded updates to have its config-data removed at the time it is downloaded from Apple and brought into reposado. I realize it shouldn't be the default as reposado is cloning the Apple SUS experience. I see the option being a key in the reposado preferences.plist to enable the feature for admins that want it.

I may be missing an obvious reason to leave the config-data intact but since cli softwareupdate doesn't see it until the config-data is removed as discussed in https://managingosx.wordpress.com/2015/01/30/gatekeeper-configuration-data-and-xprotectplistconfigdata-and-munki-and-reposado-oh-my/ the need for it seems diminished.