bucanero/pkgi-ps3

A PlayStation 3 package download tool

PKGi PS3

PKGi PS3 is a PlayStation 3 port of pkgi (PSVita).

The pkgi-ps3 homebrew app allows to download and install .pkg files directly on your PS3.

Comments, ideas, suggestions? You can contact me on Twitter and on my website.

Features

• easy to use: list available downloads, including searching, filtering, and sorting.
• standalone: no PC required, everything happens directly on the PS3.
• automatic downloads: just choose an item, and it will be downloaded by the app to your HDD (direct mode) or queued for background download (background mode) using the internal Download Manager.
• resumes interrupted downloads: you can stop a download at any time, switch applications, and come back to resume the download later.
• content activation: the app can generate .rif files for downloaded content (system must be activated)
• content updates: the app can check online for available content updates
• localization support: Finnish, French, German, Italian, Polish, Portuguese, Spanish, Turkish

Notes:

• queuing up multiple downloads is only supported when using background download mode.
• background download tasks will only show up after rebooting your PS3.

Download

Get the latest version here.

Changelog

See the latest changes here.

Setup instructions

You need to create a pkgi.txt file in /dev_hdd0/game/NP00PKGI3/USRDIR that contains the items available for installation. The text database format is user customizable. Check this section to learn how to define your own custom DB format.

Multiple databases

You can also load additional database files:

• pkgi_games.txt
• pkgi_dlcs.txt
• pkgi_themes.txt
• pkgi_avatars.txt
• pkgi_demos.txt
• pkgi_updates.txt
• pkgi_emulators.txt
• pkgi_apps.txt
• pkgi_tools.txt

Items on each of these files will be auto-categorized to the file content type. Note: The app assumes that every database file has the same format, as defined in dbformat.txt.

Online DB update

You can refresh and sync an online database by adding the DB URL(s) to the config.txt file in /dev_hdd0/game/NP00PKGI3/USRDIR.

For example:

url http://www.mysite.com/mylist.csv
url_demos http://www.demos.com/otherlist.csv
url_emulators http://www.example.com/emulators.csv

Using this setup, pkgi.txt will be updated with mylist.csv, pgi_demos.txt with otherlist.csv , and pkgi_emulators.txt with emulators.csv.

Next time you open the app, you'll have an additional menu option called Refresh. When you select it, the local databases will be syncronized with the defined URLs.

DB formats

The application needs a text database that contains the items available for installation, and it must follow the default format definition, or have a custom format definition file.

Default DB format

The default database file format uses a very simple CSV format where each line means one item in the list:

contentid,type,name,description,rap,url,size,checksum

where:

Column	Description
contentid	is the full content id of the item, for example: UP0000-NPXX99999_00-0000112223333000.
type	is a number for the item's content type. See the table below for details. (set it to 0 if unknown)
name	is a string for the item's name.
description	is a string for the item's description.
rap	the 16 hex bytes for a RAP file, if needed by the item (.rap files will be created on /dev_hdd0/exdata). Leave empty to skip the .rap file.
url	is the HTTP/HTTPS/FTP/FTPS URL where to download the .pkg file.
size	is the size in bytes of the .pkg file, or 0 if unknown.
checksum	is a SHA256 digest of the .pkg file (as 32 hex bytes) to make sure the file is not tampered with. Leave empty to skip the check.

Note: name and description cannot contain newlines or commas.

Sample DB file

An example pkgi.txt file following the contentid,type,name,description,rap,url,size,checksum format:

EP0001-FILEMANAG_00-0000000000000000,8,FileManager v1.40,File Manager,,http://github.com/Zarh/ManaGunZ/releases/download/1.40/FileManager_v1.40.pkg,12171120,FAF680636B18AD0B70AA61F48A78C5E42D6972F795F1B82CC434BE3DDE60F00F
UP0001-IRISMAN00_00-VER4880000000000,8,IRISMAN 4.88.1,Backup Manager,,http://github.com/aldostools/IRISMAN/releases/download/4.88/IRISMAN_4.88.pkg,29411984,E6EF607F0002B31BFB148BE4FC9BDBACB4E53110751F0E667C701D40B5290570
EP0001-MANAGUNZ0_00-0000000000000000,8,ManaGunZ v1.40,Backup Manager,,http://github.com/Zarh/ManaGunZ/releases/download/1.40/ManaGunZ_v1.40.pkg,17563040,CE0E4036903E881C08259FD69E777F6BC9CD24E823B471A7B15C88FDDBB2E330
UP0001-PS3SFM001_00-0000000000000000,8,Simple file manager v0.5.2,File Manager,,http://github.com/lmirel/fm_psx/releases/download/v0.5.2/sfm_ps3.pkg,1098800,301F64CC94E9BC442FDAC9199BFB8153AC2430A5E47331C6CF8A25B7881648A6
EP0001-UPDWEBMOD_00-0000000000000000,9,webMAN MOD v1.47.36,Backup Manager,,http://github.com/aldostools/webMAN-MOD/releases/download/1.47.36/webMAN_MOD_1.47.36_Installer.pkg,13580448,

Content types

Type value	Content type	DB File
0	Unknown
1	Game	pkgi_games.txt
2	DLC	pkgi_dlcs.txt
3	Theme	pkgi_themes.txt
4	Avatar	pkgi_avatars.txt
5	Demo	pkgi_demos.txt
6	Update	pkgi_updates.txt
7	Emulator	pkgi_emulators.txt
8	Application	pkgi_apps.txt
9	Tool	pkgi_tools.txt

User-defined DB format

To use a custom database format, you need to create a dbformat.txt file, and save it on /dev_hdd0/game/NP00PKGI3/USRDIR.

The dbformat.txt definition file is a 2-line text file:

• Line 1: the custom delimiter character (e.g.: ;, ,, |, etc.)
• Line 2: the column names for every column in the custom database, delimited by the proper delimiter defined in line 1

Note: For the columns to be properly recognized, use the column tag names defined in the table above.

All the columns are optional. Your database might have more (or less) columns, so any unrecognized column will be skipped.

Example

Example dbformat.txt, for a database using semi-colon (;) as separator:

;
name;TITLE ID;REGION;description;AUTHOR;TYPE;url;rap;size

Result: only the name,description,url,rap,size fields will be used.

Example

Example dbformat.txt, for a database using character pipe (|) as separator:

|
REGION|TITLE|name|url|rap|contentid|DATE|PKG FILENAME|size|checksum

Result: only the name,url,rap,contentid,size,checksum fields will be used.

Usage

Using the application is simple and straight-forward:

• Move UP/DOWN to select the item you want to download, and press .
• To see the item's details, press .
• To sort/filter/search press . It will open the context menu. Press again to confirm the new settings, or press to cancel any changes.
• Press left or right trigger buttons L1/R1 to move pages up or down.
• Press L2/R2 trigger buttons to switch between categories.

Notes

• RAP data: if the item has .rap data, the file will be saved in the /dev_hdd0/exdata/ folder.

Q&A

1. Where to get a rap string?

   You can use a tool like RIF2RAP to generate a .rap from your existing .rif files. Then you can use a tool like hexdump to get the hex byte string.

2. Where to get .pkg links?

   You can use PSDLE to find .pkg URLs for the games you own. Then either use the original URL, or host the file on your own web server.

3. Where to remove interrupted/failed downloads to free up disk space?

   Check the /dev_hdd0/tmp/pkgi folder - each download will be in a separate .pkg file by its title id. Simply delete the file and start again.

4. Download speed is too slow!

   Optimization is still pending. If direct download is slow, you can use background download mode to download files using the internal PS3 Download Manager.

Credits

• Bucanero: Project developer
• mmozeiko: PS Vita pkgi

Building

You need to have installed:

• PS3 toolchain
• PSL1GHT SDK
• Tiny3D library
• YA2D library (an extended version from my repo)
• PolarSSL library (v1.3.9)
• libcurl library (v7.64.1)
• MikMod library
• Mini18n library
• dbglogger library (only required for debug logging)

Run make to create a release build. After that, run make pkg to create a .pkg install file.

You can also set the environment variable PS3LOAD=tcp:x.x.x.x to the PS3's IP address; that will allow you to use make run and send pkgi-ps3.self directly to the PS3LoadX listener.

Debugging

To enable debug logging, build PKGi PS3 with make DEBUGLOG=1. The application will send debug messages to UDP multicast address 239.255.0.100:30000. To receive them you can use socat on your PC:

$ socat udp4-recv:30000,ip-add-membership=239.255.0.100:0.0.0.0 -

License

pkgi-ps3 is released under the MIT License.

pkgi-ps3's Issues

sha256.h: No such file or directory

I'm trying to compile PKGI but I can't, because the header file sha256.h that should be in polarssl doesn't exist

Error:

pkgi_db.c
rifrap.c
In file included from /mnt/e/Programacao/Repositorios/pkgi-ps3/source/pkgi_db.c:4:0:
/mnt/e/Programacao/Repositorios/pkgi-ps3/include/pkgi_sha256.h:5:10: fatal error: polarssl/sha256.h: No such file or directory
 #include <polarssl/sha256.h>
          ^~~~~~~~~~~~~~~~~~~
compilation terminated.
make[1]: *** [/usr/local/ps3dev/base_rules:68: pkgi_db.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:127: build] Error 2

I have already recompiled polarssl through 015-polarssl.sh but this file never appears

Any solution?

Creacion de archivos para utilizar refresh

Que tal , estaba viendo que una de las funcionalidades, es que se puede agregar un archivo para que refresque automaticamente los archivos , es posible agregar en los links desde el servidor de https://nopaystation.com/ ? que por lo menos para la vita de esa forma forma funciona

even pkgi creating the .rap files when I try to open a game I downloaded for it, a warning appears saying that I need to renew the game license to be able to play it, how do I solve this?

Fix for latest psl1ght

need ln -s /usr/local/ps3dev/portlibs/ppu/include/freetype2/freetype /usr/local/ps3dev/portlibs/ppu/include/freetype

and then git apply of:

1.txt

diff --git a/Makefile b/Makefile
index d244c30..4a8229c 100644
--- a/Makefile
+++ b/Makefile
@@ -39,7 +39,7 @@ INCLUDES	:=	include
 # any extra libraries we wish to link with the project
 #---------------------------------------------------------------------------------
 LIBS		:=	-lya2d -lfont -ltiny3d -lsimdmath -lgcm_sys -lio -lsysutil -lrt -llv2 -lpngdec -lsysmodule -lm -lsysfs  -ljpgdec -ldbglogger \
-				-lnet -lhttp -lhttputil -lssl -lfreetype -lz -lmikmod -laudio -lpolarssl
+				-lnet -lhttp -lhttputil -lssl -lfreetype -lz -lmikmod -laudio -lpolarssl -lfont3d
 
 
 #---------------------------------------------------------------------------------
diff --git a/include/pkgi_sha256.h b/include/pkgi_sha256.h
index 546ef84..e5ef46b 100644
--- a/include/pkgi_sha256.h
+++ b/include/pkgi_sha256.h
@@ -1,11 +1,11 @@
 #pragma once
 
 #include "pkgi_utils.h"
-#include <polarssl/sha2.h>
+#include <polarssl/sha256.h>
 
 #define SHA256_DIGEST_SIZE 32
 
-#define sha256_ctx			sha2_context
-#define sha256_init(ctx)	sha2_starts(ctx, 0)
-#define sha256_update		sha2_update
-#define sha256_finish		sha2_finish
+#define sha256_ctx			sha256_context
+#define sha256_init(ctx)	sha256_starts(ctx, 0)
+#define sha256_update		sha256_update
+#define sha256_finish		sha256_finish
diff --git a/source/pkgi_ps3.c b/source/pkgi_ps3.c
index 1499a04..bd6b7dc 100644
--- a/source/pkgi_ps3.c
+++ b/source/pkgi_ps3.c
@@ -696,7 +696,7 @@ void pkgi_start(void)
     sys_mutex_attr_t mutex_attr;
     mutex_attr.attr_protocol = SYS_MUTEX_PROTOCOL_FIFO;
     mutex_attr.attr_recursive = SYS_MUTEX_ATTR_NOT_RECURSIVE;
-    mutex_attr.attr_pshared = SYS_MUTEX_ATTR_PSHARED;
+    mutex_attr.attr_pshared = SYS_MUTEX_ATTR_NOT_PSHARED;
     mutex_attr.attr_adaptive = SYS_MUTEX_ATTR_ADAPTIVE;
     strcpy(mutex_attr.name, "dialog");
 

Can't download multiple dlcs

The app doesn't let you download multiple dlcs for games that use one rap file like rock band it says "Game is already Installed"

Non-ASCII characters appear as garbage on screen

Non-ASCII characters such as Japanese ones show up as random symbols on screen.

Opcion de pausar musica

Buenas , podrias agregar en un futuro update la opcion de detener la musica de fondo , ya que por mas que deje de seleccionar la opcion sigue funcionando

Idea: filter DLCs by title

I imagine it would be tricky to find out what we already have installed on the PS3, across various formats. But if we supplied a text file in the USRDIR containing a list of titles ('BLUS00001' etc), could there be an option to only show DLCs matching that list?

Cola para agregar a background

Veo que hablas español, así que creo que nos entenderemos mejor así. Pensaba en si es posible agregar la funcionalidad de una cola (queue) para agregar los contenidos a descargar al background. Con esto quiero decir que si es posible que podamos seleccionar los elementos a descargar sin tener que esperar a que lo agregue de uno en uno al background, de tal forma que se puedan seleccionar N elementos y que posteriormente los agregue al background (y se tarde las horas que tenga que tardar).

Así podriamos seleccionar por decir algo, 10 elementos de una sola vez y después dejar a la consola trabajando

Missing https support

to get online databases working you know

Can Cyrillic fonts be added?

Can I add Cyrillic fonts to display the menu in the program window?

Weird booting out sometimes

I have no idea why this happens, But it always happens when I have downloaded multiple things it’s not major but annoying

Issue downloading multi package games

Hi Bucanero,
I noticed there is an issue downloading games that are split into multiple package files. For example, pkgi is not able to download and install games if the URL for the game is a link to an XML file instead of a PKG file using either the direct or background download method. For the games that link to XML files, the direct links to the PKG files that you need to install for that game are within that XML file. I can provide examples if you need them.

If it is not possible for this to be fixed that is fine. I understand that most games are not like this and are associated with a single PKG link and not an XML file that has multiple PKG links within it.

EDIT: I just did a workaround by making my own custom file with the multiple links in it. It works fine so if you don't think this is worth addressing then it is no big deal. Thanks

PKGi not launching, only black screen

I installed pkgi on my HEN ps3 and it won't start up. I've tried cleaning the file system, but every time it just crashes to a black screen and I have to force power it off. Is there something I did wrong?

download packages directly to an ext hdd or USB

Can you look at this issue if you have the time?
Thanks :)

Some games don't download like yakuza 5 and diablo 3

Please fix that,just happen with some games and i already download all the versions but don't download at the same

Can't download anything

The program keeps showing the following error: "ERROR Could not send HTTP request"

A line example of what I tried to download:

UP9000-NPUJ01158_00-0000000000000001,0,Persona 2: Eternal Punishment (PSX),,,http://*****.pkg,461923680,

cant compile source

installed the dependencies and try to compile it i get alot of errors isk what i have wrong errors like input redined and config redifined

Startup Issue

When loading up PKGI, the error message for config/pkgi.txt shows up and none of the buttons work and nothing opens up.

How to cancel background downloads?

I selected a game to be installed on BG and since there's no indication of it's progress there's no way to cancel it. I'm on HEN and it also seems to break hen enabling sometimes. Otherwise direct download works fine.

is url_apps "http://myserver.com/PKGi/pkgi_apps.txt" invalid?

I get error: pkgi.txt file(s) missing or bad config.txt file

Rebug 4.86 Error Message

Hi,
i become a ERROR Message after i start the program.
ERROR: pkgi.txt file(s) missing or bad config.txt file.

Dlc's

Would like to request Dlc Support for the app since many games have Dlc or many Dlc's and its a hazzle to download throughout the web page

Problema abriendo pkgi

Buenas, tengo un problema al abrir el PKGI, cada vez que lo abro la consola se congela con pantalla negra y tengo que hacer hard reset, lo he desinstalado y vuelto a instalar, no logro entender por qué sucede esto, gracias.

This content can be used if you renew the licence in PlayStation Store

Any idea why I'm getting this when launching an installed game?

I'm using Jabu's NPS to PS3 to generate a TSV (with rap codes) which is read successfully by your tool (Link: https://www.psxhax.com/threads/pkgi-ps3-playstation-3-port-of-pkgi-tool-downloader-installer-by-bucanero.7210/)

But for some reason I keep getting the following error for any game I install:

This content can be used if you renew the licence in PlayStation Store

It's almost like the rap code isn't being applied? Any thoughts?

Categories not working with custom formatted txt files, odd behavior with large pkgi.txt files

bug report.zip
bug report 2.zip
I tried formatting my files per the readme.md, and they loaded in the app, but the categories function isn't working properly, (at least for me). If I tag something with the type 1/Game, it'll correctly show up in the games category (and tells me it's a game when I press [ ]), but my problem is that everything shows up in the game category regardless if it is tagged with the type 1 or any other type, which shows up as Unknown when I press Square/[ ].

Also, I have issues when I try to use more than 3 pkgi_*.txt files and the app gets stuck at 'loading pkgi.txt' and shows the database at 16384 games. I am using the NoPayStation tsvs http://nopaystation.com/ formatted with Excel to conform the the dbformat.txt that I have included in the zip. I think the problem may be that I hit the hard limit of items that can be in the database, as I have hit over 15000 items with the app actually running.

I'd be happy to do some testing on the app, but I can't compile from source due to my internet being slow.

My PS3 is a CECHH01 running FERROX 4.85.1 CFW if that helps any.

Anyways, thanks for making such a great app already!

Online db update/sync no longer working

As title says, Online db update/sync no longer works.

• for new installation nothing happens when you select the Refresh, it will just say "Refreshing 0.0kb" and just 1-2 seconds it exits out without any error and returns no results.
• for existing installation that have a cached database, online refresh makes the old database non-existent (deleting the cached list) and returns with no database at all after doing a refresh.

I tried doing my diagnosing on my part (eg Change DNS servers etc.... ) but same problem, NPS is running when doing the Online update/sync and where able to download the .tsv file on their website no issue whatsoever.

cant save rap files

after the app downloaded the .pkg, the error says it cant save .rap files. Is there anything specific needed to save .rap files

File.txt

Hi! I installed this new version but it does not show, do I need the txt? If I do, where can I find it?

(Background Mode) No funciona

Hey bucanero, he probado a descargar con la opción Background mode y al hacer click en un juego no descarga en segundo plano, sino que se descarga desde alli entero

Que puede ser?

[Suggestion] NoPayStation specific setting on the app.

It would be nice if the app supports NPS out of the box (well this is PKGi that is based on Vita that is mainly purpose is to work with NPS and download stuff based on NPS database).

A dedicated settings for this to enable it and manually enter the URL links to the TSV files. And doesnt require a separate setting like dbformat.txt for it to be able to read it.

tl;dr a GUI support to setup NPS without the nerdy steps to enable it (eg edit config.txt and create your own dbformat.txt to read nps tsv files).

Already Installed Error when installing DLC

As title says, install a DLC shows a Already Installed error.

how to trigger:

• Install a game data of any game and search for DLC on PKGi, it will result on Already Installed error
• It can also trigger by just installing the game update as well (if the game data install is optional and have an update)
• It can also trigger if the digital version of the game is already installed and you want to install a DLC for the same game.
• It can also trigger if there's a existing DLC is installed of the same game.
• For some reason, even just creating a empty folder with same gameid inside the game folder on dev_hdd0 of the DLC will also trigger the already installed issue.

*well the apps is false flagging it was already installed when in reality its not as long as it detects the same GAMEID

the only solution for me this now is deleting the game data for game disc or the game it self if its a digital version, which can be annoying.

images

Error "Could not send HTTP Request"

As the title says, it intermittently pops-up an error "Could not send HTTP Request"

• it happens randomly on a very busy network.
• I think it happens with slow DNS server

The only fix for this now is keep retrying the download until it proceeds, sometimes it only requires 2 tries sometimes as far as 10 tries. I think pkgi should have be doing the retrying for a several times before popping out the error.

[Question] Is there planned support for ****Station.com?

http://****station.com/

pkgi-ps3 can't create "exdata" directory.

When installing a game with the rap included in the .txr. file, it shows up an error if the exdata directory doesn't exist, you have to manually create it for the raps to be saved properly.

Selected title jumps to the first item in the list

I can press down and page down but as soon as I stop the selection jumps to the first title in the list. If I am fast I can install an item before it jumps.

Can not change downloading option to background download

Everytime I open the settings dialog it shows the downloading option is "Direct. DL". I change it to "Back. DL", close the settings dialog then reopen it immediately, it shows "Direct. DL" again.

install does not actually install

Pressing X to install a package does not actually install it, it just downloads it. I suggest renaming this to download or actually installing the package.

In addition, please if you could change the UI so that the O button closes the side menu instead of resetting it, I always forget to press Triangle. Normally O represents back in everything.

[Suggestion] only show categories which are not empty

The categories like Emulators, Tools when selected, at least for me show games, it is because they are empty. It would be better if they were not even selectable if there is no corresponding pkgi_x.txt file, or if the default file had nothing for that category.

In addition to this, the All category could possibly have an indicator as to what type each item is, reason being, say I wanted to display a game and all its DLCs in one place, I would see them under all but not know which is the game and which is the theme and which is the DLC.

Program doesn't ignore commas inside quotes

If you have a game name with commas, the program will treat the comma as the CSV delimiter, regardless if it´s in quotes, as seen at

pkgi-ps3/source/pkgi_db.c

Line 197 in 961ec41

while (ptr < end && *ptr != ',')

podrias hacer PKGI compatible con las bases de datos de esta web?

buenas, ¿podrias hacer que PKGI fuese compatible con las bases de datos de esta web?

http://nopaystation.com

Background Download Not working

Background Download Mode does Not Work if more than one package is queued. Nothing appears in the Package/PSN section

Freezing and black screen when opening PKGi on PS3Hen 4.87

When opening PGKi on PS3Hen 4.87 there is a general crash and a black screen, which can only be fixed with forced console shutdown. Any tips?

Thanks!

Character corruption in Japanese

By this happen, very hard to seach jpn games

Socat don't receive messages

I already installed socat compiling directly from the source,
and I also put the DEBUGLOG=1 flag in the make command, and before starting pkgi through PS3Load I start socat using socat udp4-recv: 30000, ip-add-membership = 239.255.0.100: 0.0.0.0 -, however when starting pkgi I don't get anything on the console, I suspect it's Windows (because I'm using Ubuntu 20.04.2 for WSL 1), but honestly I don't know, any ideas?

The console looks like this:

Program freezes if there's an invalid URL

Whenever there's something that can't be interpreted as an URL such as the "MISSING" value after converting dbs to pkgi-ps3, the program freezes, but can be exitted using the PS Button.

"Already installed" DLC's issue

I can install every other DLC, but if i have an installed and updated game, it shows that all the DLC's available for that game are installed when is not.

Improve updater code

Improve auto-updater code to support HTTPS and download PKGi updates directly from GitHub's repo.

Filtro para juegos instalados

Paso una idea que estaría bueno agregar en un futuro , que al igual que tiene el pkgj de vita , poder filtrar por juegos que uno ya tiene instalados ( diferenciados por region ) para que al momento de querer instalar un DLC no tener que buscar entre los miles que se encuentran e instalar el correspondiente a la region del juego para que funcione