dolkensp/unp4k

Unp4k utilities for Star Citizen

unp4k

These tools allow users to open, decrypt, and extract data from Star Citizen .p4k files.

Installation:

1. Download unp4k-suite-v3.3.x.zip
2. Right click the zip and select Properties
3. Under the General tab, check the Unblock checkbox at the bottom if it exists, and click ok
4. Extract the selected zip to your desired installation directory

NOTE: unforge currently requires .Net Framework 4.6.2 or greater to run. If you receive the error Method not found: '!!0[] System.Array.Empty()'. you will need to install this framework.

Quickstart:

1. Drag Data.p4k from the Starcitizen\LIVE folder, directly onto unp4k.exe

Advanced Command Line Usage:

1. Launch a command line, and navigate to the unp4k directory
2. Execute unp4k.exe c:\path\to\data.p4k [filter] where filter is a keyword used to filter the results (the default filter is *.*)

NOTE: The filter does not fully support wildcards. To extract files of a certain type, you may use *.ext as the filter, but no further wildcard functionality exists.

Basic GUI Usage:

1. Launch unp4k.gui.exe
2. Select File > Open and browse to your chosen Data.p4k
3. Browse the file structure
4. Right click to extract/open files

NOTE: unp4k.gui is early alpha, and has many crashes, and unfinished features. Use at your own risk

File Format Overview:

The p4k files used by Star Citizen are Zip archives.

Star Citizen supports data in multiple modes inside the archive, including STORE, DEFLATE, and custom support for ZSTD.

Star Citizen also implements bespoke encryption over some of the data inside the archive - this can all be decrypted with the same public key that is utilized by CryEngine based games for various encryption routines within the engine.

Inside the p4k file, XML files are often stored as CryXML rather than raw XML.

CryXML is a basic serialized XML format which unforge.exe is able to deserialize.

Inside the p4k file, there is also a game.dcb file, which is a bespoke database format, with similarities to CryXML.

This is the product of what is known internally as "DataForge", and is also able to be converted/extracted using the unforge.exe tool.

unp4k's Issues

the source fro ICSharpCode.TreeView does not match the provided dll

the source fro ICSharpCode.TreeView does not match the provided dll
could you please upload the updated sources.

Compile in Linux

Is unp4k possible to compile in Linux? if 'yes' any hints?

No biggie if not possible. I understand its an fringe use case.

.socpak files

Hi Peter,

Have you got any idea about how to extract information from .socpak files? I think for larger spaceships they split some of the interior into their own object containers. I'm guessing SOCPAK = Streaming Object Container PAK.

Specifically, I would love to be able to explore the contents of these Carrack related files because I think some of the loadout information is in there.

Data\ObjectContainers\Ships\ANVL\Carrack\front.socpak
Data\ObjectContainers\Ships\ANVL\Carrack\middle.socpak
Data\ObjectContainers\Ships\ANVL\Carrack\rear.socpak

error converting Star citizen 2.5 game.dcb

Unable to unforge the game.dcb located in the Data/Datacore.pak in Star Citizen 2.5.

Log is as attached..
log.txt

3.14

don't think unp4kGUI.exe is compatible, only gets a few files then crashes

Application do not extract folder properly

I noticed when trying to extract several folders or a specific folder, it don't create the folder when extracting. Instead, it extract the folder files on the root folder.

MAUI Implementation

I've made this so the MAUI implementation can be discussed.

I'm also wanting feedback from @peter-dolkens as to if unp4k should use MAUI Native or the MAUI Blazor framework.
MAUI Native is faster and has less overhead but only for the GUI side of things yet, it has much less capabilities because of how new it is.
As for MAUI Blazor, it has the overhead of Server-Client and of course Chromium and the fact we will use SASS and Typescript BUT our only limitation is Chromium + we will be able to use our already existing web knowledge.

3.23 Exported character file analysis

Hello!

I'd like to start off by saying that this isn't strictly about unp4k, but I hope it's close enough that you'll have some insight I do not, or at least point me towards a better place for me to investigate.

With the 3.23 EPTU patch, we can now save character files as *.chf. Some examples here: https://www.star-citizen-characters.com/

According to CIG, these files are encrypted and then compressed. Out of curiosity I dissected some of these files and am gathering research here: https://github.com/diogotr7/StarCitizenChf

So far I've managed to dissect the header for the file, the crc checksum check and the compression (zstd).

Trying to figure out the encryption seems like a much bigger can of worms to deal with which is why I'm making this issue.

I see unp4k has a hardcoded key in the code when unpacking crypack(?) files. Is this something that is standard to all cryengine games? Is this something you had to reverse engineer out of the game? Do you have any advice for ways I could decrypt this?

Thanks for any insight, if you feel this is too offtopic of course feel free to close.

No mount is added to the cart when selecting Ball S4 to the Hornet Ghost

The Anvil F7C-S Hornet Ghost comes with no element to fit anything on the top-center slot of the ship.
Currently, when selecting 'Ball 2xS2', the website add the C4-160f S5 Hornet Ball Turret to the cart, which is good.
However, when selecting 'Ball S4', The Specialty Hornet S5 VariPuck Mount is not added to the cart.

help finding some game values

Hi, first of all, thank you very much for your work with this tool.

I've been fiddling around trying to reverse engineer these files for values like commodity prices and stock and I've managed to do so, but I'm now trying to find values for these and can't find anything:

• ships' cargo capacity
• store position values(so that I can measure distances)

I was wondering if this is just not included or if anyone could have found this already and could point me in the right direction. Thanks

More info on p4k file?

@peter-dolkens I am trying to figure out how to open this p4k file myself, but I'm not succeeding like you seem to be. I see you have some kind of encryption key in your unzip code, but what kind of encryption is it? I'm not familiar with zip so I am having to rely heavily on libs doing this for me. I am trying to do it in rust, but I'm not sure what the differences are between this file and a regular zip. I was wondering if you could elaborate on the file format a bit more than the readme already contains?

Also, weirdly, when I use 7zip to open it I only see about 6 files, all related to a starfarer crash...

Unable to unforge game.dcb

Warning Tag.eb6425f0-f7a0-4a1d-a4b1-a2ed73d5ce56 doesn't match libs/foundry/records/tagdatabase/tagdatabase.tagdatabase.xml
Warning Tag.5f4faa96-65fb-4771-a59f-3c316b866302 doesn't match libs/foundry/records/tagdatabase/tagdatabase.tagdatabase.xml
Warning Tag.336ec2d5-eda4-42da-b001-927d4685c4b9 doesn't match libs/foundry/records/tagdatabase/tagdatabase.tagdatabase.xml
Warning Tag.13a35f23-6ef5-4de3-888b-32f8995c6910 doesn't match libs/foundry/records/tagdatabase/tagdatabase.tagdatabase.xml
Warning Tag.614a7309-3041-4ead-b00c-19b0f6e51d11 doesn't match libs/foundry/records/tagdatabase/tagdatabase.tagdatabase.xml
Warning Tag.cfd28b7d-b3a7-49c4-834f-42a2714b0f8d doesn't match libs/foundry/records/tagdatabase/tagdatabase.tagdatabase.xml
Warning UnitTestSubRecord.UnitTestSubRecordA doesn't match libs/foundry/records/unittest.unittestb.xml
Warning UnitTestSubRecord.UnitTestSubRecordB doesn't match libs/foundry/records/unittest.unittestb.xml
Warning ScreenEffects_Effect.2bd09f5c-4645-43ba-9b3b-db1b0a47009f doesn't match libs/foundry/records/vfx/generalscreeneffects.xml
Warning ScreenEffects_Effect.4761f510-f7bc-4af9-8eb2-92b012a976c0 doesn't match libs/foundry/records/vfx/generalscreeneffects.xml
Warning ScreenEffects_Effect.e42cea73-2429-407e-9622-5682d8a44d8a doesn't match libs/foundry/records/vfx/generalscreeneffects.xml
Warning ScreenEffects_Effect.a56fc94d-91f9-4adc-803d-2f63e345e252 doesn't match libs/foundry/records/vfx/generalscreeneffects.xml
Warning ScreenEffects_Effect.9138f15c-6e77-450c-8976-151b8e542297 doesn't match libs/foundry/records/vfx/generalscreeneffects.xml
Warning ScreenEffects_Effect.e38b3a54-491d-45a9-8140-8eab35d87488 doesn't match libs/foundry/records/vfx/generalscreeneffects.xml
Warning ScreenEffects_Effect.523ecff4-29ea-4ea7-94f5-2cb307f22b6a doesn't match libs/foundry/records/vfx/generalscreeneffects.xml
Warning ScreenEffects_Effect.9736fbf9-eb33-497c-b95a-1392803a5058 doesn't match libs/foundry/records/vfx/generalscreeneffects.xml
Warning VoiceSingle.VoiceSingle7 doesn't match libs/foundry/records/voicebundle/bhuntpilot01.xml
Warning VoiceSingle.CruPatrol1 doesn't match libs/foundry/records/voicebundle/crupatrol.xml

When i try to run unforge.exe Game.dcb i get these errors, how can i run unforge on the game.dcb no files are generated from this.

Out of memory exception

Tried to run precompliled and compiled by myself, always getting out of memory exception after choosing the file and little waiting:

System.OutOfMemoryException
at System.Threading.ThreadPool.QueueUserWorkItemHelper(System.Threading.WaitCallback, System.Object, System.Threading.StackCrawlMark ByRef, Boolean)
at System.Threading.ThreadPool.UnsafeQueueUserWorkItem(System.Threading.WaitCallback, System.Object)
at System.Threading.TimerQueue.QueueTimerCompletion(System.Threading.TimerQueueTimer)
at System.Threading.TimerQueue.FireNextTimers()
at System.Threading.TimerQueue.AppDomainTimerCallback(Int32)

System.OutOfMemoryException
HResult=0x8007000E
Message=Eine Ausnahme vom Typ "System.OutOfMemoryException" wurde ausgelöst.
Source=Microsoft.VisualStudio.DesignTools.WpfTap
StackTrace:
at Microsoft.VisualStudio.DesignTools.WpfTap.WpfVisualTreeService.VisualTree.RootSourceFinder.OnHwndParentChange(IntPtr hWinEventHook, UInt32 eventType, IntPtr hwnd, Int32 idObject, Int32 idChild, UInt32 dwEventThread, UInt32 dwmsEventTime)

Reimplementation and modernization of parts of unp4k

Hello, I'm making this issue as a thanks for your work mostly, and also as a way to let you know I've decided to port unp4k and unforge to modern c# versions, making them faster and hopefully cleaner since i have your work to reference.

Here's your changes to SharpZipLib required to extract unp4k's, with clean git history and rebased on the newest SharpZipLib codebase: https://github.com/diogotr7/SharpZipLibP4k

Here's my rewrite of unp4k's DataForge decoder, p4k extractor, and hopefully some sort of gui at some point in the future? https://github.com/diogotr7/StarBreaker

Again, thank you for all your findings, i had some trouble reimplementing things with the reference, I cannot image how head scratching it must have been the first time around :)

Unable to Unforge and unpack Game.dcb since PTU 3.13

Hi,

I love your tool and I use it constantly :) . However, since PTU 3.13 I am not able anymore to unpack the Game.dcb. What am I doing wrong? It seems to me that there is an adjustment required in the tool?

Kind Regards

i wanna do localization work

Sorry to disturb you,i just have three questions.
1: Can i replace the language file "Data\Localization\english\global.ini" ?
2: If i replace it , other language which i was write in file can display in game?
3: I only konw how to unpack it ,but i don't kown how to replace file in data.p4k or repack all files.

Changes to Support Datacore V5

Recent changes to the DCB has caused Unforge to fail with the following
Error converting Game.dcb: The output char buffer is too small to contain the decoded characters, encoding 'Unicode (UTF-8)' fallback 'System.Text.DecoderReplacementFallback'. Parameter name: chars

Application closing when extracting specific folder

First, I´d like to thank you for this very handy tool!

I'm using the latest version (3.3.65) on an up-to-date Windows 10 (64-bit, 1803). Latest SC too (3.3.7-LIVE.1007767).

Every time that I try to directly extract the following folder, all files are extracted, but right in the end the program closes, before returning control to the treeview:

Data > Objects > landscape > rock > Alpha

All other directories seem to work fine,

Repack p4k

This is mostly a feature request, from the description it doesn't show option for repacking the file, if it is there then you can just ignore this.
I'd love to see the option to repack the files uncompressed, I wonder if it will speed up game loading on some faster SSD's and reduce CPU usage, especially on NVME drives.