SPSS, PSPP php implementation (read/write .sav files)
License: MIT License
A PHP library for reading and writing SPSS / PSPP .sav data files.
composer require tiamo/spss
Please have a look in CHANGELOG
Licensed under the MIT license.
First of all thank you for this great library. It has been of great help to us.
We are using GNU PSPP 1.4.1. We get warning for all fields width. SPSS considers width greater than 255 to be long string record. But at the moment, every field is being listed in very long string record.
This is a code snippet from Sav/Writer.php around line 179. Apparently the Record\Variable::isVeryLong($var->width) is expected to return boolean value if we look at the implementation. Because of this every field is being marked as very long string.
if (Record\Variable::isVeryLong($var->width) !== 0) {
$this->info[Record\Info\VeryLongString::SUBTYPE][$shortName] = $var->width;
}
I replace 0 with false and could get rid of the warnings. But when using width above 255, We get blank file on GNU PSPP. Because of this we are bound to use width below 255.
Has anyone had any luck getting this package to work with multi byte names?
If I fix the regex to allow it I get errors like
error: `/run/user/1000/doc/e43f6ab4/test.sav' near offset 0x9f640: Unrecognized record type 1443443282.
The names include danish letters like æøå/ÆØÅ which are valid in SPSS variable names. Example from pspp
I have tried making my own version where I replaced all php methods to the mb_ version (like mb_strlen), but sadly it does not seem to work. :/
I'm getting errors on SPSS 23 that are resolved if I remove the EOF opcode.
Attached a file that seems to open just fine, but then when trying to analyze it (for example **descriptives > frequencies **) gives:
Incomplete (truncated) SPSS Statistics data file: path
The attached data file contains 1 case, 2 variables. This is the smallest way I could reproduce the error.
Would you accept a PR that removes the EOF opcode functionality?
When I set valuelabel, variablelable and data in Chinese, the value of the field in the sav file is Mojibake.
I can't get the library to export actual zero values for variables with the format Variable::FORMAT_TYPE_F. Instead, the library is converting all 0 values to blanks when writing the SPSS file. Here is some sample code that will illustrate the point.
$file = '<path>'
$data = [
'header' => [
'creationDate' => '01 01 2020',
'creationTime' => '01:01:01',
],
'variables' => [
[
'name' => 'TEST_VAR',
'format' => Variable::FORMAT_TYPE_F,
'width' => 40,
'decimals' => 2,
'label' => 'TEST LABEL',
'values' => [],
'columns' => 5,
'alignment' => Variable::ALIGN_RIGHT,
'measure' => Variable::MEASURE_SCALE,
'attributes' => [
'$@Role' => Variable::ROLE_INPUT,
],
'data' => [1,0,2,3,4,0,5],
]
],
];
$writer = new Writer($data);
$writer->save($file);
If I write this to a file the responses with '0' as the value will be blank, not the number 0. I've tried adding the following:
'values' => ['0' => '0'],
and
'values' => ['0.00' => '0'],
I've tried other variations with and with out quotes and some end up creating a corrupt SPSS file and the others didn't work as well. I've also tried making sure the zeroes in the data array are entered as 0.00 but that did not work either.
I'm imagine I must be missing something but this appears to be a bug. Thanks in advance for your time!
Hi, i am a bit lost here. How can I get variable format in reader? for example I want to know if the data is datetime, string or numeric?
Hi,
Probably I'm doing something wrong but in some cases, my exported sav file causes crash on spss and gives the following error message on PSPP:
Variable index n refers to long string continuation.
Could you help me where should I start the investigation?
How do I define the missing value (SYSMIS) in my data set?
The library is working great for me. However it appears that whenever I try to define a FORMAT_TYPE_A (string) variable with a width of over 255 characters, the SAV file does not open and crashes SPSS.
SPSS should support strings up to 32767 characters.
https://www-01.ibm.com/support/docview.wss?uid=swg21476969
Hi!,
I have problem with save variable like below. Scripts is hung and only max execution time can stop script. Can You check that?
Probably its by polish character "ł" in label, when replace that with "l" its ok.
BUT When
$variable->label = 'latały';
export ok (same polish character as above)
Can You help?
Code:
$variable = new Variable();
$variable->name = 'P2';
$variable->label = 'Z jakich uslug instytucji fintechowych korzystałes/korzystasz? - Ubezpieczeniowe np. zakup ubezpieczen przez Facebooka lub polisomacie, zakup ubezpieczen wg zuzycia krotkoterminowe AC, zgloszenie szkodu z wykorzystaniem mediow spolecznosciowych Facebook Messenger Warta, telematyka';
$writer = new Writer([
'header' => [
'prodName' => 'Spss',
],
'variables' => [$variable],
]);
I'm using the example write.php from the master branch, to generate the file data.sav. When i try to open the file as it's generated in the PSPP software (GNU pspp version 1.0.1), the file can not be opened and i received the following output:
warning: `/home/lestcape/Documentos/aaa/data.sav' near offset 0x18c: File
specifies unexpected value -1.79769e+308 (-0x1.fffffffffffbp+1023) as SYSMIS,
instead of -1.79769e+308 (-0x1.fffffffffffffp+1023).
warning: `/home/lestcape/Documentos/aaa/data.sav' near offset 0x18c: File
specifies unexpected value 1.79769e+308 (0x1.fffffffffffbp+1023) as HIGHEST,
instead of 1.79769e+308 (0x1.fffffffffffffp+1023).
warning: `/home/lestcape/Documentos/aaa/data.sav' near offset 0x18c: File
specifies unexpected value -1.79769e+308 (-0x1.fffffffffffbp+1023) as LOWEST,
instead of -1.79769e+308 (-0x1.ffffffffffffep+1023) or -1.79769e+308 (-0x1.
fffffffffffffp+1023).
error: `/home/lestcape/Documentos/aaa/data.sav' near offset 0xdc: Missing
string continuation record.
As most of warnings are related with the writer info i remove the comments for that session:
// 'info' => [
// 'machineInteger' => [
// 'machineCode' => 720,
// 'version' => [23, 0, 0],
// ],
// 'machineFloatingPoint' => [
// 'sysmis' => -1.7976931348623157e+308,
// 'highest' => 1.7976931348623157e+308,
// 'lowest' => -1.7976931348623155e+308,
// ],
// ],The new resulting file is also not opened. I have not warnings now, but the error remaining and i get the following output after execute the example:
error: `/home/lestcape/Documentos/aaa/data.sav' near offset 0xdc: Missing
string continuation record.
Note:
When I remove the variable bbbb_bbbbbb12 from the writer, the resulting data.sav file can be opened in pspp. So, the problem seem to be related with something there.
I am using this library to write an SPSS file. The data I am using is a very reasonable size. There are 100 variables and 1000 responses. All questions are floats with a value of 1 or 0 for the response. I have bumped the php max execution time to 15 minutes and it still is timing out. Through my logs I know that it takes four seconds to get the data, process it, and send it to the Writer so I am certain it's the Writer that is responsible for the long execution time. Is this expected behavior? Is there something I can do to speed this process up?
My variable names are being trimmed to 8 chars, causing numerous duplicate variable names. It seems this 8 char limit is coded pretty deeply into the project ( $buffer->writeString($this->name, 8); ).
Is there a flag or setting that I am missing to extend the name length to 64 chars (available since SPSS version 12)?
If it will require code changes could you give a high-level summary of the implications and scope of making this change?
Thanks for you time!
I tried the test_writer output on v20, and it could not read the file. It causes an error. Has there been any changes in the the file format in later versions of spss?
The test_read can however read the samples it came with the v20.
Thanks in advance.
Here is my code:
$file="test16.sav";
$data = [
'header' => [
'prodName' => '@(#) SPSS DATA FILE test',
'layoutCode' => 2,
'compression' => 1,
'weightIndex' => 0,
'bias' => 100,
'creationDate' => '01 01 2020',
'creationTime' => '01:01:01',
],
'info' => [
'machineFloatingPoint' => [
'sysmis' => -1.7976931348623157e+308,
'highest' => 1.7976931348623157e+308,
'lowest' => -1.7976931348623155e+308,
],
],
'variables' => [
[
'name' => 'VAR1',
'format' => \SPSS\Sav\Variable::FORMAT_TYPE_A,
'width' => 40,
'label' => 'LABEL1',
'values' => ["A1"=>"男" ,"A2"=>"女"],
'columns' => 8,
'alignment' => \SPSS\Sav\Variable::ALIGN_RIGHT,
'measure' => \SPSS\Sav\Variable::MEASURE_SCALE,
'attributes' => [
'$@ROLE' => \SPSS\Sav\Variable::ROLE_INPUT,
],
'data' => ["A1","A2"],
],
[
'name' => 'VAR2',
'format' => \SPSS\Sav\Variable::FORMAT_TYPE_A,
'width' => 255,
'label' => 'LABEL2',
'values' => ["A1A1A1A1A1A1A1A1A1A1A1"=>"男男男男男男男男男男男男" ,"A2A2A2A2A2A2A2A2A2A2A2"=>"女女女女女女女女女女女"],
'columns' => 8,
'alignment' => \SPSS\Sav\Variable::ALIGN_RIGHT,
'measure' => \SPSS\Sav\Variable::MEASURE_SCALE,
'attributes' => [
'$@ROLE' => \SPSS\Sav\Variable::ROLE_INPUT,
],
'data' => ["A1A1A1A1A1A1A1A1A1A1A1","A2A2A2A2A2A2A2A2A2A2A2"],
]
],
];
$writer = new \SPSS\Sav\Writer($data);
$writer->save($file);
When I opened it using PSPP, I found that the value label was garbled or empty，May I ask how to solve it?
I have the following input:
$input = [
'header' => [
'prodName' => '@(#) SPSS DATA FILE - Example',
'creationDate' => '25 Jul 19',
'creationTime' => '21 07 49',
'weightIndex' => 0,
'compression' => 0,
],
'variables' => [
[
'alignment' => 1,
'attributes' => ['$@Role' => 3],
'columns' => 16,
'decimals' => 0,
'format' => 22,
'label' => "Interview date",
'measure' => 3,
'name' => "interview_date",
'width' => 20,
'data' => [
13783456875
]
],
[
'alignment' => 1,
'attributes' => ['$@Role' => 3],
'columns' => 10,
'decimals' => 0,
'format' => 5,
'label' => "Do you want to continue?",
'measure' => 3,
'name' => "consent",
'values' => [
0 => 'No',
1 => 'Yes'
],
'width' => 9,
'data' => [
'1'
]
]
]
];
The resulting .sav file make SPSS freeze for ever, it won't open.
There are two ways to make the file usable:
Reverse the variables. If the "consent" variable comes first, then everything is okay, the file will open just fine.
Remove the "values" (value labels) from the "consent" variable. Again, if those are removed, the file will open just fine.
My project's specs is such that I'm not allowed to reverse the order of the variables and I also need to support value labels, so could you please help me identify the problem?
I found a bug in the library.
When you look at the demo from test_write.php and switch the Numeric and String codeblocks. So create strings vars first and then create numeric vars. SPSS will crash when trying to open the .sav file.
When the value of the value label is a string, the generated spss file value label is incomplete.
This is an example
array (
'name' => 't1',
'format' => 1,
'width' => 2,
'label' => 'test spss: test spss',
'values' => array(
'01' => '1000',
'02' => '2000',
'03' => '3000',
'04' => '4000',
),
'columns' => 2,
'alignment' => 0,
'measure' => 1,
'attributes' => array(
'$@Role' => 0,
),
'data' => array(),
)
Value should be
01 = "1000"
02 = "2000"
03 = "3000"
01 = "4000"
But now it is like this
Could you please explain how to make "missingValue" work.
I tried:
$var->missingValueFormatCode = 0;
$var->missingValues = array(-99);
But the above code doesn't work.
I need to make a sav file with around 200 variables and 7500 cases (this is the largest export currently).
The problem is that it is currently taking around 10-11 minutes to create the file on my dev machine. I fear it will be even slower on production.
Has anyone found a clever way of speeding it up?
I was hoping to somehow split it up running parallel but sadly merging files seems quite hard (I know the files could be merged in spss later but I have been asked to just deliver a single spss file that can be downloaded and used).
I have encountered difficulties in exporting Date and Time Formats data. Can anyone help me please?thx.
Write large variables ( array ) with appending mode should save large memory ?
Hi again!
When I set PL chars in the label of variable or text in values in a variable, saved .sav file in SPSS have trimmed text.
Example, when i set:
$variable->label = 'Data zakończenia';
In SPSS i got label "Data zakończeni".
This same is when I set example:
$variable->values = [ 1 => 'Kobieta', 2 => 'Mężczyzna' ];
In SPSS i got "Mężczyzn" rather then "Mężczyzna"
Can You check this?
Hi,
who happen to meet this error when trying to read save file?
{ "message": "unpack(): Type i: not enough input, need 4, have 0", "exception": "ErrorException", "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/flobee/spss/src/Buffer.php", "line": 439, "trace": [ { "function": "handleError", "class": "Illuminate\\Foundation\\Bootstrap\\HandleExceptions", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/flobee/spss/src/Buffer.php", "line": 439, "function": "unpack" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/flobee/spss/src/Buffer.php", "line": 317, "function": "readNumeric", "class": "SPSS\\Buffer", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/flobee/spss/src/Sav/Record/Info/LongStringValueLabels.php", "line": 21, "function": "readInt", "class": "SPSS\\Buffer", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/flobee/spss/src/Sav/Record.php", "line": 31, "function": "read", "class": "SPSS\\Sav\\Record\\Info\\LongStringValueLabels", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/flobee/spss/src/Sav/Record/InfoCollection.php", "line": 56, "function": "fill", "class": "SPSS\\Sav\\Record", "type": "::" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/flobee/spss/src/Sav/Reader.php", "line": 152, "function": "fill", "class": "SPSS\\Sav\\Record\\InfoCollection", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/flobee/spss/src/Sav/Reader.php", "line": 105, "function": "readBody", "class": "SPSS\\Sav\\Reader", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/app/Http/Controllers/FileController.php", "line": 92, "function": "read", "class": "SPSS\\Sav\\Reader", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Routing/Controller.php", "line": 54, "function": "getDownloadTmpFile", "class": "App\\Http\\Controllers\\FileController", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php", "line": 45, "function": "callAction", "class": "Illuminate\\Routing\\Controller", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Routing/Route.php", "line": 239, "function": "dispatch", "class": "Illuminate\\Routing\\ControllerDispatcher", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Routing/Route.php", "line": 196, "function": "runController", "class": "Illuminate\\Routing\\Route", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 685, "function": "run", "class": "Illuminate\\Routing\\Route", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 128, "function": "Illuminate\\Routing\\{closure}", "class": "Illuminate\\Routing\\Router", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Auth/Middleware/EnsureEmailIsVerified.php", "line": 29, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Illuminate\\Auth\\Middleware\\EnsureEmailIsVerified", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php", "line": 41, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Illuminate\\Routing\\Middleware\\SubstituteBindings", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php", "line": 78, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Illuminate\\Foundation\\Http\\Middleware\\VerifyCsrfToken", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php", "line": 49, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Illuminate\\View\\Middleware\\ShareErrorsFromSession", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php", "line": 116, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php", "line": 62, "function": "handleStatefulRequest", "class": "Illuminate\\Session\\Middleware\\StartSession", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Illuminate\\Session\\Middleware\\StartSession", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php", "line": 37, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Illuminate\\Cookie\\Middleware\\AddQueuedCookiesToResponse", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php", "line": 67, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Illuminate\\Cookie\\Middleware\\EncryptCookies", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 103, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 687, "function": "then", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 662, "function": "runRouteWithinStack", "class": "Illuminate\\Routing\\Router", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 628, "function": "runRoute", "class": "Illuminate\\Routing\\Router", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 617, "function": "dispatchToRoute", "class": "Illuminate\\Routing\\Router", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php", "line": 165, "function": "dispatch", "class": "Illuminate\\Routing\\Router", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 128, "function": "Illuminate\\Foundation\\Http\\{closure}", "class": "Illuminate\\Foundation\\Http\\Kernel", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php", "line": 21, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php", "line": 21, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php", "line": 27, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/CheckForMaintenanceMode.php", "line": 63, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Illuminate\\Foundation\\Http\\Middleware\\CheckForMaintenanceMode", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/fruitcake/laravel-cors/src/HandleCors.php", "line": 37, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Fruitcake\\Cors\\HandleCors", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/fideloper/proxy/src/TrustProxies.php", "line": 57, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 167, "function": "handle", "class": "Fideloper\\Proxy\\TrustProxies", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 103, "function": "Illuminate\\Pipeline\\{closure}", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php", "line": 140, "function": "then", "class": "Illuminate\\Pipeline\\Pipeline", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php", "line": 109, "function": "sendRequestThroughRouter", "class": "Illuminate\\Foundation\\Http\\Kernel", "type": "->" }, { "file": "/Users/mohdaidilrahmanshakri/Development/htdocs/nihdatarepo/public/index.php", "line": 55, "function": "handle", "class": "Illuminate\\Foundation\\Http\\Kernel", "type": "->" } ] }
I figured out my issue. The problem is that there is a bug in the code and it guarantees that any variable with a label that has a length longer than 255 will cause an infinite while loop. The code in the Record\Variable class at line 147 is this:
while ($labelLengthBytes > 255) {
// Strip one char, can be multiple bytes
$label = mb_substr($this->label, 0, -1);
$labelLengthBytes = mb_strlen($label, '8bit');
}You can see that $this->label never changes so the loop will go on forever. Here is the fix:
while ($labelLengthBytes > 255) {
// Strip one char, can be multiple bytes
$label = mb_substr($label, 0, -1);
$labelLengthBytes = mb_strlen($label, '8bit');
}The fix is to update line the "$this->label" to "$label" on line 149 so the label length actually decreases in length. If someone from the team can confirm my suspicions are correct I can submit a quick PR.
Hi. I'm using your library... It's amazing useful for read spss files directly, but i don't use the writer, as actually pspp can generate the spss from a postgres database in a low level, pretty faster. A big thanks for the reader then. I experimented some crashes with some specific sav files in the version 1, but not with the version 2. The problem it's that: In the version 2 apparently, you re-wrote almost all for scratch, and understand the version 2 was a little hard to me. I think because is without an example... Then, will be a good for promote your library, if you add again the same examples of version 1 but with version 2.
Speaking about particular things, i think will be good create a function to detected missing values in an easy way, (A way to self contained the most common tasks). The system missing values are difficult to implement if you don't have any idea of the internally representation of system missing values.
I think, will be useful rearrange the info index ordering to have into account the extended values, as the variables have not the same index because contains the extended values. I don't found a good way to detected the extended value, i just ask for not empty name then.
I continues testing your good work, for now i can not found any big problem... Version 2 have a better performance than version 1... I don't know about memory usage yet, but i think this could be maybe the only possible problem. You remove the loadData function and you are loading the data when you constructed the object. In a possible scenario that we have a big table of data and we only want to read the header and not the data, will not be a good idea load all data.
Again this is amazing and thanks for this library.
Is it possible to edit .sav file by its base64 data and download it to client browser without saving it to server
I have a game file name as active.sav and when I try to edit it with your library then it's give a error why this error is happening
I have a working implementation to write MR Sets, but this is not PR ready.
Anyone interested in implementing a PR for this?
In the tests/SavTest.php, we need to add require_once '../vendor/autoload.php'; to work it out. : )
I am writing data with ValueLables. The data can be opened alright, but the labels are invalid. The value portion of the label does not appear, rendering the value column on spss as invalid.
Thanks
The below code will create a .sav file that can be opened in SPSS. However, if I change the width from 255 to 256 the resulting file cannot be opened in SPSS.
$data = [
'header' => [
],
'variables' => [
[
'name' => 'TEST',
'format' => Variable::FORMAT_TYPE_A,
'width' => 255, // <- if this is 256 the .sav file cannot be opened
'decimals' => 0,
'label' => 'LABEL',
'values' => [],
'missing' => [],
'columns' => 15,
'alignment' => Variable::ALIGN_LEFT,
'measure' => Variable::MEASURE_NOMINAL,
'attributes' => [
'$@Role' => Variable::ROLE_INPUT,
],
'data' => ['test'],
],
]
];
$writer = new Writer($data);
$writer->save('test.sav');If I try to open it, here is the SPSS error I'm getting:
GET
FILE='/<path>/test.sav'.
>Error. Command name: GET FILE
>Invalid SPSS Statistics data file: /<path>/test.sav (DATA1204)
>Execution of this command stops.
>Error # 1405 in column 8. Text: /<path>/test.sav
>Error when attempting to get a data file.
DATASET NAME DataSet1 WINDOW=FRONT.
Actually we are reading/writing the whole data file in a matrix at once and this is not ideally when we have big files. One idea is provide an API to set the buffer in the position where the case start and read that case only. Of course when the file is compressed all cases size are different, so... This will required just read the position in the file where the case is, instead of read the real case at once. Then on demand, we can go to that position in the file and read the case that we want.
Please note: If the data is not compress we don't need to iterate the file on the load, as all cases have the same size, so we can predict where the case start.
$casePosition = //Is an array that will be filled when the data is loaded, instead of fill the matrix. Then if we want just a case:
$buffer->setPosition($casePosition[$case]);
return $buffer->readCase(); //This is an array.and we can iterate for all cases if we want the whole file:
for ($case = 0; $case<count($casePosition); $case++) {
$buffer->setPosition($casePosition[$case]);
$matrix[$case] = $buffer->readCase();
}
return $matrix;To write the file is also useful that can be done case by case instead of the whole info at once.
Hi!
When save file with below config, i cannot open saved file in SPSS - application hangs.
BUT, saved file opens success when:
-remove all variable 1 or 2
OR
-remove "width" from first variable
OR
-remove "values" from second variable
Anyone can help me? What i do wrong?
My code:
$writer = new \SPSS\Sav\Writer([
'header' => [
'prodName' => '@(#) ' . $this->analysis->getName(),
'layoutCode' => 2,
'compression' => 1,
'weightIndex' => 0,
'creationDate' => date('Y-m-d'),
'creationTime' => date('H:i:s'),
],
'variables' => [
[
'name' => 'aaa',
'width' => 16,
'format' => 1,
],
[
'name' => 'cc',
'format' => 5,
'values' => [
1 => 'Panel',
]
],
]
]);
$pathFile = 'spss.sav';
$writer->save($pathFile);
Hi,
Like in title. Can You help ASAP?
Thx!
I am trying to write a sav file, but it seems that this regex does not allow #? That should be valid character in spss names?
https://github.com/tiamo/spss/blob/master/src/Sav/Writer.php#L115C31-L115C67
From an existing .sav file
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.