xeokit / xeokit-convert Goto Github PK
View Code? Open in Web Editor NEWConvert various AEC model formats for efficient viewing in the browser with xeokit.
Home Page: https://xeokit.github.io/xeokit-convert/docs/
License: Other
Convert various AEC model formats for efficient viewing in the browser with xeokit.
Home Page: https://xeokit.github.io/xeokit-convert/docs/
License: Other
Released in @xeokit/xeokit-convert 1.0.9
convert2xkt
automatically partitions each model's objects into tiles.
The vertex coordinates of the objects within each tile may then be stored in XKT as single-precision values that are relative to the center of their tiles. XKT also compresses the coordinates to integers, by quantizing them to an unsigned 16-bit integer range that maps to the extents of their tiles.
As ````convert2xkt``` converts each object, it selects a tile to place it in, converting the object's coordinates to RTC (relative-to-center-of-tile) and quantizing them to the tile boundary.
We lose accuracy in the following case:
This is because, within the IEEE754 floating point standard, the availability of precise floating point values becomes more sparse as the magnitude of the value increases.
Ultimately, it makes sense to automatically select the tile size as a function of this magnitude.
For now, however, we'll expose a new mintilesize
option to convert2xkt
.
node convert2xkt.js -h
Usage: convert2xkt [options]
Options:
-v, --version output the version number
-s, --source [file] path to source file
-f, --format [string] source file format (optional); supported formats are gltf, ifc, laz, las, pcd, ply, stl and cityjson
-m, --metamodel [file] path to source metamodel JSON file (optional)
-i, --include [types] only convert these types (optional)
-x, --exclude [types] never convert these types (optional)
-r, --rotatex rotate model 90 degrees about X axis (for las and cityjson)
-g, --disablegeoreuse disable geometry reuse (for ifc and gltf)
-t, --mintilesize [number] minimum diagonal tile size (optional, default 1000)
-o, --output [file] path to target .xkt file; creates directories on path automatically if not existing
-l, --log enable logging
-h, --help display help for command
convert2xkt.js -s ./Railway.json -o Railway.xkt -t 2000 -l
[convert2xkt] Reading input file: ./Railway.json
[convert2xkt] Input file size: 4521.41 kB
[convert2xkt] Converting...
[convert2xkt] Converting CityJSON 1.0
[convert2xkt] Converted to: XKT v9
[convert2xkt] XKT size: 846.34 kB
[convert2xkt] Compression ratio: 5.34
[convert2xkt] Conversion time: 2.44 s
[convert2xkt] Converted metaobjects: 123
[convert2xkt] Converted property sets: 0
[convert2xkt] Converted drawable objects: 120
[convert2xkt] Converted geometries: 36984
[convert2xkt] Converted triangles: 113537
[convert2xkt] Converted vertices: 170281
[convert2xkt] reuseGeometries: true
[convert2xkt] minTileSize: 2000
[convert2xkt] Writing XKT file: /Railway.xkt
glTF materials define a "doubleSided" property, which disables back-face culling (see https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#double-sided)
Use that property to decide whether parsed triangles meshes should be of xeokit type "solid" or "surface".
How can I convert a IFC file to XKT with a MetaModel JSON? This is a little problem...
convert2xkt
glTF->XKT conversion mode breaks when the glTF file provides its geometry in a separate binary file attachment.
This can be reproduced by running the open source converter tools like this:
IfcConvert-v0.6.0-517b819-linux64/IfcConvert ./myModel.ifc ./myModel.dae
COLLADA2GLTF/build/COLLADA2GLTF-bin -i ./myModel.dae -o ./myModel.gltf -s
convert2xkt.js -s ./myModel.gltf -o ./myModel.xkt -l
convert2xkt
uses fs.readFileSync
to load the binary file, and expects the loaded data to be an ArrayBuffer
. However, that function returns a Buffer
.
Solution is to simply convert the Buffer
to an ArrayBuffer
.
I'm following the steps here: https://www.notion.so/Converting-IFC-Models-to-XKT-using-3rd-Party-Open-Source-Tools-c373e48bc4094ff5b6e5c5700ff580ee
The steps all worked without any errors however the coordinates when viewed in the bim-viewer website seem a little off. My models have large coordinates (e.g. ~6,000,000,000mm and ~300,000,000mm in Sydney, Australia)
Shown in this screenshot are pipes and pipe fittings and as you can see they don't line up.
Thoughts?
Fixed in npm 1.0.0-beta.7
Test (give this a couple of minutes to convert and load): https://xeokit.github.io/xeokit-convert/visualTests/test_convert_PLY_Crankcase.html
XKT version: 9
node version: v14.17.0
Converted a file to STL with Blender 2.79
[convert2xkt] Reading input file: house.stl
[convert2xkt] Input file size: 123.48 kB
[convert2xkt] Converting...
TypeError: First argument to DataView constructor must be an ArrayBuffer
at new DataView ()
at isBinary (/home/####/node_modules/@xeokit/xeokit-convert/dist/convert2xkt.cjs.js:54156:20)
at /home/####/node_modules/@xeokit/xeokit-convert/dist/convert2xkt.cjs.js:54131:13
at new Promise ()
at parseSTLIntoXKTModel (/home/####/node_modules/@xeokit/xeokit-convert/dist/convert2xkt.cjs.js:54090:12)
at convert (/home/####/node_modules/@xeokit/xeokit-convert/dist/convert2xkt.cjs.js:64016:13)
at convertForFormat (/home/####/node_modules/@xeokit/xeokit-convert/dist/convert2xkt.cjs.js:64000:21)
at /home/####/node_modules/@xeokit/xeokit-convert/dist/convert2xkt.cjs.js:63916:13
at new Promise ()
at convert2xkt (/home/####/node_modules/@xeokit/xeokit-convert/dist/convert2xkt.cjs.js:63844:12)
Hi @xeolabs,
I'm wondering if a stable release is coming up soon, as multiples dependencies have been bumped and some due to security issues. I know that this is an open source project and as such I'm not really asking for a very precise ETA, but could this come in a month or ? Maybe few months or weeks ?
This fixes an incompatibility within xeokit-sdk
an xeokit-bim-data
with the metadata JSON and glTF created by IFC2GLTFCxConverter_v2.1.0_210616
("converter").
For each IFC element that has a geometric representation (eg. IfcDoor), the converter currently creates a composite of objects, in which the root object has an ID that matches the IFC element, and its child objects represent its parts (eg. frame and panel). The child objects have random IDs that don't match any IFC elements, and the same type as their root.
This confuses xeokit-sdk
and xeokit-bim-viewer
, which expects every object in a model to correspond to an IFC element.
In this fix, convert2xkt
automatically combines such composite objects into single objects, that have IDs that match IFC elements.
The screenshot below shows the result of this combination. Using a TreeViewPlugin configured to be in "containment" hierarchy mode, we've made a single object made visible, which is an IfcDoor that contains a frame and panel, which are both separate meshes within the viewer. Toggling visibility on the IfcDoor, for example, will toggle both the frame and panel in unison.
Within the metadata created by IFC2GLTFCxConverter_v2.1.0_210616
, that object was a represented as a composite of three metaobjects having type IfcDoor
, in which the root had an IFC UUID, and the children had random IDs. By combining that, we're able to represent the whole door as one object in the tree view.
If we call myViewer.scene.setObjectsSelected(["<IFCDoor element ID>"], true)
, then that will select our object and set both the panel and frame to appear selected.
I have problems with IFCSpace when I convert IFC files in xkt v9.
A metamodel is created, but there are no entity or “normal” objects created in the viewer.
I also downloaded the example xkt files “Duplex.ifc.xkt” in version v8 (https://github.com/xeokit/xeokit-sdk/blob/master/assets/models/xkt/v8/ifc/Duplex.ifc.xkt) and v9 (https://github.com/xeokit/xeokit-sdk/blob/master/assets/models/xkt/v9/ifc/Duplex.ifc.xkt) to check my problem.
When I load the v8 into the viewer I get the IFCSpaces (as visible entities) but in the v9 not (only metamodels).
I try to install as said in https://www.notion.so/Converting-Models-to-XKT-with-convert2xkt-fa567843313f4db8a7d6535e76da9380
git clone https://github.com/xeokit/xeokit-convert.git
cd xeokit-convert
npm install
Installation throw errors
D:\Trabajo\NPM\BIM\xeokit-convert\xeokit-convertPS>npm install
npm WARN deprecated @percy/[email protected]: PercyScript has been deprecated in favor of Percy CLI's snapshot command: https://docs.percy.io/docs/percy-snapshot Please upgrade to use the CLI rather than this package, as it's no longer actively supported.
npm WARN deprecated [email protected]: request-promise has been deprecated because it extends the now deprecated request package, see request/request#3142
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-node-resolve.
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: jsSHA versions < 3.0.0 will no longer receive feature updates
npm WARN deprecated [email protected]: This module moved to @hapi/sntp. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: request has been deprecated, see request/request#3142
npm WARN deprecated [email protected]: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-commonjs.
npm WARN deprecated [email protected]: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic.
See https://v8.dev/blog/math-random for details.
npm WARN deprecated [email protected]: request has been deprecated, see request/request#3142
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated [email protected]: This module moved to @hapi/hawk. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.
npm WARN deprecated [email protected]: support for ECMAScript is superseded by uglify-js
as of v3.13.0
npm WARN deprecated [email protected]: This package is unmaintained and deprecated. See the GH Issue 259.
npm WARN deprecated [email protected]: Firefox support is gradually transitioning to the puppeteer package. As of puppeteer v2.1.0 you can interact with Firefox Nightly. The puppeteer-firefox package will remain available until the transition is complete, but it is no longer actively maintained. For more information visit https://wiki.mozilla.org/Remote
npm WARN deprecated [email protected]: core-js@<3.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
npm WARN deprecated [email protected]: core-js@<3.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
npm ERR! code 1
npm ERR! path D:\Trabajo\NPM\BIM\xeokit-convert\xeokit-convert\node_modules\puppeteer-firefox
npm ERR! command failed
npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node install.js
npm ERR! ERROR: Failed to download Firefox rv0.0.1!
npm ERR! Error: Download failed: server returned code 404. URL: https://github.com/puppeteer/juggler/releases/download/v0.0.1/firefox-win64.zip
npm ERR! at D:\Trabajo\NPM\BIM\xeokit-convert\xeokit-convert\node_modules\puppeteer-firefox\lib\BrowserFetcher.js:264:21
npm ERR! at ClientRequest.requestCallback (D:\Trabajo\NPM\BIM\xeokit-convert\xeokit-convert\node_modules\puppeteer-firefox\lib\BrowserFetcher.js:320:7)
npm ERR! at Object.onceWrapper (node:events:646:26)
npm ERR! at ClientRequest.emit (node:events:526:28)
npm ERR! at HTTPParser.parserOnIncomingClient (node:_http_client:618:27)
npm ERR! at HTTPParser.parserOnHeadersComplete (node:_http_common:128:17)
npm ERR! at TLSSocket.socketOnData (node:_http_client:482:22)
npm ERR! at TLSSocket.emit (node:events:526:28)
npm ERR! at addChunk (node:internal/streams/readable:315:12)
npm ERR! at readableAddChunk (node:internal/streams/readable:289:9)
npm ERR! -- ASYNC --
npm ERR! at BrowserFetcher. (D:\Trabajo\NPM\BIM\xeokit-convert\xeokit-convert\node_modules\puppeteer-firefox\lib\helper.js:32:15)
npm ERR! at Object. (D:\Trabajo\NPM\BIM\xeokit-convert\xeokit-convert\node_modules\puppeteer-firefox\install.js:47:16)
npm ERR! at Module._compile (node:internal/modules/cjs/loader:1103:14)
npm ERR! at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)
npm ERR! at Module.load (node:internal/modules/cjs/loader:981:32)
npm ERR! at Function.Module._load (node:internal/modules/cjs/loader:822:12)
npm ERR! at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
npm ERR! at node:internal/main/run_main_module:17:47
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\garenovich\AppData\Local\npm-cache_logs\2022-04-22T14_54_44_386Z-debug-0.log
Although this is supported in direct IFC->XKT conversion mode, it's missing for this workflow.
I'm trying this command:
$ node ./node_modules/@xeokit/xeokit-convert/convert2xkt.js -s foo.ifc -o hyd.xkt -l
[convert2xkt] Reading input file: foo.ifc
[convert2xkt] Input file size: 48258.68 kB
[convert2xkt] Converting...
web-ifc: 0.0.34 threading: 0
Error: ${err}
Any ideas how I can help debug?
Note: that was installed via npm i @xeokit/xeokit-convert
If I clone from main and npm install
then get this instead:
$ node convert2xkt.js -s hyd.ifc -o hyd.xkt -l
[convert2xkt] Reading input file: hyd.ifc
[convert2xkt] Input file size: 48258.68 kB
[convert2xkt] Converting...
web-ifc: 0.0.34 threading: 0
Error: TypeError: FromRawLineData[rawLineData.type] is not a function
Add option to provide JSON file of pre-parsed IFC metadata as input to convert2xkt
, alongside an IFC file, so that the tool only parses geometry from the IFC, and parses metadata from the JSON instead of the IFC.
Please add string encoding support
https://technical.buildingsmart.org/resources/ifcimplementationguidance/string-encoding/
Implementation examples:
https://github.com/opensourceBIM/IfcPlugins/blob/11420abed00f4ab5843d7b75ac54dbf39649c6fe/IfcPlugins/src/org/bimserver/ifc/step/deserializer/IfcParserWriterUtils.java#L173
https://github.com/xBimTeam/XbimEssentials/blob/master/Xbim.Common/Step21/XbimP21StringDecoder.cs
I had several issues with converting a las-file to xkt. I'm neither an expert for node.js nor for 3d-file-formats or ifc. Our goal is to load an ifc-file and a point cloud scan of the built model into the same viewer. Unfortunately, I can't provide the raw input data. Both files are in the same coordinate system. If I open the raw data with blender both models are on top of each other and everything is fine.
I did some edits directly in dist/convert2xkt.cjs.js to get the conversion to work:
a) (see #26 ): I just added "typeof Response !== 'undefined'" to some ifs at various places. (main issue was in function validHTTPResponse(), somewhere around line 50960. Can this ever be defined within a CLI-tool?)
b) the --rotatex parameter is not used for LAS-files, only for LAZ. (there is some big switch-block with various file formats in function convert2xkt, somewhere around line 83500.) I don't know if this was intentional or just forgotten, but for my point cloud it didn't work without it. From my understanding, LAS and LAZ should be basically the same, LAZ has just better compression?
c) the entire point cloud was mirrored. To fix this I changed the coordinate shifting in function readPositions (around line 71450) to positionsValue[i + 2] = temp * -1; If this is, depending on the input-data, an issue in some cases, but not in all, maybe this should be another cli parameter?
Not sure if this is helpful, but with these changes, so far, everything seems to be working fine for me.
Add an option to split convert2xkt
output into multiple XKT files.
What to split on?
I have this issue with node script convert2xkt.js where the options where not recognised (e.g. I set "-s filename" and script acts as if it's not set).
In convert2xkt.js we access the options through "program." (e.g. program.source) and that doesn't work. I changed that to "options.source" and all the other "program.xxx" to "options.xxx" too, and now it works.
Funny thing is that is works as it is on windows but it didn't work on ubuntu. I would still suggest changing it to "options." instead of "program." as in the commander docs.
Seems like it's a bug or maybe I am doing something wrong?
Use loaders.gl to help with glTF parsing.
This has the following benefits:
.glb
fix this https://github.com/xeokit/xeokit-gltf-to-xkt/issues/39
(node:23672) UnhandledPromiseRejectionWarning: Parameter expected: params.primitiveType
(Use node --trace-warnings ...
to show where the warning was created)
(node:23672) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict
(see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:23672) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
This one will be a bit technical, just grab your 🍿!
isTriangleMeshSolid
functionBy looking at the code, flagSolidGeometries
is invoking isTriangleMeshSolid
method here...
This function seems to do the following:
It scans all edges in a certain triangle-defined surface and, if each edge is used exactly twice, it considers the surface as watertight. This means that the surface completely encloses the volume in such a way that water inside the volume could not escape though any hole in the surface (thus watertight).
It is considered that each triangle in the geometry defines exactly 3 edges: 1st=>2nd vertex indices; 2nd => 3rd vertex indices; 3rd => 1st vertex indices.
In order to count each edge is found exactly twice in the surface, it need to count edges.
Problem 1: in order to count how many times each edge is used, it uses a string hash key
composed of...
{edge vertex index 1}-{edge vertex index 2}
... and then it iterates all the triangles in the geometry, using a dictionary as auxiliary structure for counting how many times each edge is used.
The problem is that this generates a massive number of string keys (plus insertion in a dictionary), and the number of generated string hashes is num-triangles-in-model * 3
.
In order for 2.
to work well, the code needs to make sure any different vertices that have the same position coordinates are effectively considered the same, so it has a pre-processing stage to make sure that e.g. two indices referring to same coordinate (in case of duplicate vertices) are re-mapped the same index. Otherwise 2.
would not work.
Problem 2: an auxiliary dictionary is used to "uniquify" the vertices, and this again involves the generation of a tremendous amount of string keys. In this case the key is composed of...
{vertex-pos-X},{vertex-pos-Y},{vertex-pos-Z}
... and the number of keys generated is exactly the same as the number of total vertices in the model.
Even though isTriangleMeshSolid
only uses temporary dictionaries (both in step 2.
and 3.
) to do its job, the massive amount of strings generated and stored in dictionaries is causing the Javascript Engine to suffer from performance and RAM issues, due to (I guess) the extensive usage of the GC to free the dictionaries plus the strings after each execution of isTriangleMeshSolid
.
For example, the following RAM measurement method has been from nodejs
before/after executing _flagSolidGeometries
...
let usedMb = process.memoryUsage().heapUsed / 1024 / 1024;
(https://nodejs.org/api/process.html#processmemoryusage)
... and for a moderate IFC file (around 126 MB), the used memory by the process increases by around 107 MB after executing flagSolidGeometries
.
I suppose the question at this point is: is there any way to count duplicate items (as this is used both in 2.
to count edge occurrences and in 1.
to remap indices corresponding to unique vertex positions) but avoiding the usage of RAM-hungry data structures?
(and this includes dictionaries of course if they need string hashes per-counted-item)
Well, here I propose you a conceptually very easy solution. The reasoning is the following:
What if, starting from a list of items, we could sort the list to semantically same items get grouped together? Then, in the counting phase, iterate over the sorted list knowing that if two items must be considered the same, they will be adjacent in the list!
With this solution, if we stick to JS .sort(...)
method (which does sorting in-place), we don't need to allocate any additional memory for auxiliary structures 😃!
One natural objection to the proposed approach is that:
O(N)
O(N · log N)
So the natural question is the performance would be much affected by the proposed algorithm.
I just measured with a really small IFC file (9 MB) and a moderate one (126 MB), and the results are the following for flagSolidGeometries
method:
So, apart from reducing execution time, the solution is MUCH more RAM friendly, which is all about the topic of this issue.
And, finally, the code! ⚙️
Sorry not to create a PR, preferred to include the code here with the comments above:
You will notice two additional arguments are supplies to the new isTriangleMeshSolid
method:
vertexIndexMapping
and edges
Those are used just to avoid to allocate big lists per processed geometry!
In order to generate those two arguments, add the following to the beginning of _flagSolidGeometries
method:
_flagSolidGeometries() {
let maxNumPositions = -1;
let maxNumIndices = -1;
for (let i = 0, len = this.geometriesList.length; i < len; i++) {
const geometry = this.geometriesList[i];
if (geometry.primitiveType === "triangles") {
if (geometry.positionsQuantized.length > maxNumPositions) {
maxNumPositions = geometry.positionsQuantized.length;
}
if (geometry.indices.length > maxNumIndices) {
maxNumIndices = geometry.indices.length;
}
}
}
let vertexIndexMapping = new Array (maxNumPositions / 3);
let edges = new Array (maxNumIndices);
// ...
geometry.solid = isTriangleMeshSolid2(
geometry.indices,
geometry.positionsQuantized,
vertexIndexMapping,
edges
); // Better memory/cpu performance with quantized values
// ...
}
The previous code will make sure to pre-allocate big enough arrays to be used as temporary lists in the new isTriangleMeshSolid
code. (Their size can't be statically determined as it depends on the processed model).
const isTriangleMeshSolid = (indices, positions, vertexIndexMapping, edges) => {
function compareIndexPositions(a, b)
{
let posA, posB;
for (let i = 0; i < 3; i++) {
posA = positions [a*3+i];
posB = positions [b*3+i];
if (posA !== posB) {
return posB - posA;
}
}
return 0;
};
// Group together indices corresponding to same position coordinates
let newIndices = indices.slice ().sort (compareIndexPositions);
// Calculate the mapping:
// - from original index in indices array
// - to indices-for-unique-positions
let uniqueVertexIndex = null;
for (let i = 0, len = newIndices.length; i < len; i++) {
if (i == 0 || 0 != compareIndexPositions (
newIndices[i],
newIndices[i-1],
)) {
// different position
uniqueVertexIndex = newIndices [i];
}
vertexIndexMapping [
newIndices[i]
] = uniqueVertexIndex;
}
// Generate the list of edges
for (let i = 0, len = indices.length; i < len; i += 3) {
const a = vertexIndexMapping[indices[i]];
const b = vertexIndexMapping[indices[i+1]];
const c = vertexIndexMapping[indices[i+2]];
let a2 = a;
let b2 = b;
let c2 = c;
if (a > b && a > c) {
if (b > c) {
a2 = a;
b2 = b;
c2 = c;
} else {
a2 = a;
b2 = c;
c2 = b;
}
} else if (b > a && b > c) {
if (a > c) {
a2 = b;
b2 = a;
c2 = c;
} else {
a2 = b;
b2 = c;
c2 = a;
}
} else if (c > a && c > b) {
if (a > b) {
a2 = c;
b2 = a;
c2 = b;
} else {
a2 = c;
b2 = b;
c2 = a;
}
}
edges[i+0] = [
a2, b2
];
edges[i+1] = [
b2, c2
];
if (a2 > c2) {
const temp = c2;
c2 = a2;
a2 = temp;
}
edges[i+2] = [
c2, a2
];
}
// Group semantically equivalent edgdes together
function compareEdges (e1, e2) {
let a, b;
for (let i = 0; i < 2; i++) {
a = e1[i];
b = e2[i];
if (b !== a) {
return b - a;
}
}
return 0;
}
edges = edges.slice(0, indices.length);
edges.sort (compareEdges);
// Make sure each edge is used exactly twice
let sameEdgeCount = 0;
for (let i = 0; i < edges.length; i++)
{
if (i === 0 || 0 !== compareEdges (
edges[i], edges[i-1]
)) {
// different edge
if (0 !== i && sameEdgeCount !== 2)
{
return false;
}
sameEdgeCount = 1;
}
else
{
// same edge
sameEdgeCount++;
}
}
if (edges.length > 0 && sameEdgeCount !== 2)
{
return false;
}
// Each edge is used exactly twice, this is a
// watertight surface and hence a solid geometry.
return true;
};
What do you think @xeolabs? 😃
When I execute
node convert2xkt.js -h
I get an error
node:internal/modules/cjs/loader:936
throw err;
^
Error: Cannot find module 'commander'
Require stack:
Removing this because
(More info soon)
XKT v10-alpha
format,convert2xkt
to extract all the material types from glTF and encode them into XKT using this new formatXKTLoaderPlugin
to load XKT 10-alpha
PerformanceModel
and its renderer) to store and render texturesTo start with, add these low-level fields to the XKT format:
uvs
array, which holds all UVseachGeometryUVsPortion
arraytextures
array, which holds data for all texturesTo map each mesh to an arbitrary set of textures in the texturesArray
.
eachMaterialType
- for each material, it's type: 0 = custom, 1 = metallic/rough PBR, 2 = specular/rough PBR, 3 = Blinn/Phong etc.eachMaterialTextures
- for each material, an arbitrary-length array of indices of textures
used by that material - interpretation of what the textures depends on material typeeachMaterialAttributes
- for each material, an arbitrary-length array of floats - interpretation depends on material typeThen we re-purpose the existing eachMeshMaterial
field, making it index the eachMaterialXXX
arrays, instead of defining per-mesh material colors inline.
Some additions to the JS programming API within xeokit-convert
:
XKTPhongMaterial
, XKTMetalRoughMaterial
and XKTSpecularRoughMaterial
classesXKTModel#createMaterial
methodmaterialId
param to XKTModel#createMesh
methodXKTModel#materials
mapWhen converting IFC files exported from VectorWorks 20 we get the following error:
Debugger listening on ws://127.0.0.1:9229/ae17d9a7-733d-4ea3-be1f-05c87148462e
For help, see: https://nodejs.org/en/docs/inspector
web-ifc: 0.0.29 threading: 0
Error: TypeError: Cannot read property 'value' of null
at /home/gidoca/src/third_party/xeokit/xeokit-convert/xeokit-convert/dist/convert2xkt.cjs.js:48846:81
at Array.forEach (<anonymous>)
at parseRelatedItemsOfType (/home/gidoca/src/third_party/xeokit/xeokit-convert/xeokit-convert/dist/convert2xkt.cjs.js:48844:25)
at parseSpatialChildren (/home/gidoca/src/third_party/xeokit/xeokit-convert/xeokit-convert/dist/convert2xkt.cjs.js:48792:5)
at /home/gidoca/src/third_party/xeokit/xeokit-convert/xeokit-convert/dist/convert2xkt.cjs.js:48848:21
at Array.forEach (<anonymous>)
at parseRelatedItemsOfType (/home/gidoca/src/third_party/xeokit/xeokit-convert/xeokit-convert/dist/convert2xkt.cjs.js:48844:25)
at parseSpatialChildren (/home/gidoca/src/third_party/xeokit/xeokit-convert/xeokit-convert/dist/convert2xkt.cjs.js:48784:5)
at /home/gidoca/src/third_party/xeokit/xeokit-convert/xeokit-convert/dist/convert2xkt.cjs.js:48848:21
at Array.forEach (<anonymous>)
The reason appears to be that the file contains lines like the following:
#66033=IFCRELDEFINESBYPROPERTIES('01b$bsqtH0eBDv9j4xF5mW',#1,$,$,($),#64425);
Specifically ($)
seems to cause problems here (and does not really make sense, but still ends up in real world files). This commit on my fork contains a hack that allows the file to run through.
Create new convert2xkt
option r
, to rotate LAS and CityJSON models 90 degrees about the X-axis as they are converted.
node convert2xkt -h
Usage: convert2xkt [options]
Options:
-v, --version output the version number
-s, --source [file] path to source file
-f, --format [string] source file format (optional); supported formats are gltf, ifc, laz, las, stl and cityjson
-m, --metamodel [file] path to source metamodel JSON file (optional)
-i, --include [types] only convert these types (optional)
-x, --exclude [types] never convert these types (optional)
-r, --rotatex rotate model 90 degrees about X axis (for las and cityjson)
-o, --output [file] path to target .xkt file; creates directories on path automatically if not existing
-l, --log enable logging
-h, --help output usage information
XKT version: 9
To rotate a sample model:
node convert2xkt -s assets/models/laz/indoor.0.1.laz -o indoor.0.1.laz.xkt -r -l
Possibilities:
Convert every N points
For each n points, output an average point
Option to specify a maximum file size / memory size / point number, so we can ensure the XKT will fit in browser memory
Geometry instancing is the practice of rendering multiple copies of the same mesh in a scene at once.
WebGL rendering performance can really suffer when the number of instanced meshes is too high, even if the number of instances of each mesh is still low.
For this reason, we need the option to disable geometry instancing for convert2xkt
and parseGLTFIntoGeometry
.
This would "expand" instanced meshes, creating duplicate copes within the .xkt
. The penalty is a larger .xkt
file, where the size increase depends on how geometry reuse happens in the model.
Add a disablegeoreuse
(-g
) option to make convert2xkt
disable geometry reuse while converting.
Also add info on this new option to the tool's help info:
node convert2xkt.js -h
Usage: convert2xkt [options]
Options:
-v, --version output the version number
-s, --source [file] path to source file
-f, --format [string] source file format (optional); supported formats are gltf, ifc, laz, las, pcd, ply, stl and
cityjson
-m, --metamodel [file] path to source metamodel JSON file (optional)
-i, --include [types] only convert these types (optional)
-x, --exclude [types] never convert these types (optional)
-r, --rotatex rotate model 90 degrees about X axis (for las and cityjson)
-g, --disablegeoreuse disable geometry reuse (for ifc and gltf)
-o, --output [file] path to target .xkt file; creates directories on path automatically if not existing
-l, --log enable logging
-h, --help display help for command
XKT version: 9
node convert2xkt.js \
-s IFC_Schependomlaan.gltf \
-m IFC_Schependomlaan.json \
-o IFC_Schependomlaan.xkt -l -g
if the IFC has a conversion measure unit - it is not updated in the xkt model
i.e. :
#43= IFCSIUNIT(*,.LENGTHUNIT.,$,.METRE.);
#44= IFCDIMENSIONALEXPONENTS(1,0,0,0,0,0,0);
#45= IFCMEASUREWITHUNIT(IFCRATIOMEASURE(0.3048),#43);
#46= IFCCONVERSIONBASEDUNIT(#44,.LENGTHUNIT.,'FOOT',#45);
expected:
viewer.scene.metrics.units === 'feet'
actual:
viewer.scene.metrics.units === 'meters'
At the time of conversion, some models (large and small) had elements left out.
Using the pattern described by bimspot presents the correct elements as per the modeler.
Is there any specific configuration to solve this problem?
Link for model Link
node convert2xkt.js -s HID-R00.ifc -o HID-R00.xkt -l
...
XKTEntity has no meshes - won't create: 0C$jkpSpT1e9D_WKWEOFDV
XKTEntity has no meshes - won't create: 0C$jkpSpT1e9D_WKWEOF1G
XKTEntity has no meshes - won't create: 0C$jkpSpT1e9D_WKWEOFUf
XKTEntity has no meshes - won't create: 2QR6j3eqD1nBRYB9JI5DfZ
XKTEntity has no meshes - won't create: 2QR6j3eqD1nBRYB9JI5Dfk
XKTEntity has no meshes - won't create: 2QR6j3eqD1nBRYB9JI5DfR
XKTEntity has no meshes - won't create: 2QR6j3eqD1nBRYB9JI5Dhf
XKTEntity has no meshes - won't create: 2QR6j3eqD1nBRYB9JI5DhK
XKTEntity has no meshes - won't create: 2fTRVXJyfDtgdyqgs_GApO
XKTEntity has no meshes - won't create: 2fTRVXJyfDtgdyqgs_GAp6
XKTEntity has no meshes - won't create: 1H7Rle_gX6VukYdFGRTXRs
XKTEntity has no meshes - won't create: 0$80N_8mz639VZV5SqE5kl
XKTEntity has no meshes - won't create: 1gZfGABc1B7e4mLF2w0Tcr
XKTEntity has no meshes - won't create: 3kEVZD8cHFGOx3b1jmWCUJ
[convert2xkt] Converted to: XKT v9
[convert2xkt] XKT size: 956.79 kB
[convert2xkt] Compression ratio: 27.50
[convert2xkt] Conversion time: 82.68 s
[convert2xkt] Converted metaobjects: 1647
[convert2xkt] Converted property sets: 23082
[convert2xkt] Converted drawable objects: 4961
[convert2xkt] Converted geometries: 732
[convert2xkt] Converted triangles: 37916
[convert2xkt] Converted vertices: 81394
[convert2xkt] reuseGeometries: true
[convert2xkt] minTileSize: 1000
[convert2xkt] Writing XKT file: HID-R00.xkt
How can I convert a IFC file with more than 2GB?
(node:5620) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use node --trace-warnings ...
to show where the warning was created)
D:\lib\xeokit-convert\src\convert2xkt.js:1
import {parseMetaModelIntoXKTModel} from "./parsers/parseMetaModelIntoXKTModel.js";
^^^^^^
SyntaxError: Cannot use import statement outside a module
at Object.compileFunction (node:vm:352:18)
at wrapSafe (node:internal/modules/cjs/loader:1026:15)
at Module._compile (node:internal/modules/cjs/loader:1061:27)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1151:10)
at Module.load (node:internal/modules/cjs/loader:975:32)
at Function.Module._load (node:internal/modules/cjs/loader:822:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
at node:internal/main/run_main_module:17:47
Node.js v17.6.0
Dependencies are installed:
xeokit-convert>npm install --only=prod
operating system:
Windows 10, 64 bit
When converting IFC models from glTF and JSON metadata into XKT, the property sets are missing from the XKT.
This bug occurs when converting glTF+JSON like so:
node convert2xkt.js -s Schependomlaan.gltf -m Schependomlaan.json -o Schependomlaan.xkt
Property sets are also incorrectly described within the XKT v9 schema specification. This fix also fixes the specification, as described below.
On each metaobject, the spec incorrectly defines a property "propertySetId", which is a string. That property should instead be an array of strings, called "propertySetIds".
Incorrect schema spec:
"metaObjects": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"type": {
"type": "string"
},
"id": {
"type": "string"
},
"parent": {
"type": "string"
},
"propertySetId": {
"type": "string"
}
},
"required": [
"name",
"type",
"id"
]
}
]
}
Corrected schema spec:
"metaObjects": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"type": {
"type": "string"
},
"id": {
"type": "string"
},
"parent": {
"type": "string"
},
"propertySetIds": {
"type": "array",
"items": [
{
"type": "string"
}
]
}
},
"required": [
"name",
"type",
"id"
]
}
]
}
When I started testing, we had problems restoring packages. I emphasize here the caution for the release and warnings about possible related problems.
Installing with npm i @xeokit/xeokit-convert
Breakage when bringing package
Cloning Release
Missing files like convert2xkt.js
Download the Release in ZIP format
At the time of restoring the package, some dependencies are discontinued and with errors in the download. The packages are:
puppeteer-firefox
@percy/script
Another point of care is the versions of dependencies, although xeokit-convert was released at the end of April, we still have some packages that have become obsolete.
Package Avalible Latest
@loaders.gl/core 3.0.13 3.2.3
@loaders.gl/json 3.0.13 3.2.3
@loaders.gl/las 3.0.13 3.2.3
@loaders.gl/obj 3.2.3 3.2.3
@loaders.gl/ply 3.2.3 3.2.3
@loaders.gl/polyfills 3.2.3 3.2.3
@zip.js/zip.js 2.2.38 2.4.12
autoprefixer: 9.8.5 10.4.7
earcut 2.2.2 2.2.3
jszip 3.6.0 3.10.0
puppeteer 10.4.0 14.1.2
request 2.79.0 2.88.2
web-ifc 0.0.30 0.0.35
xmldom 0.6.0 0.6.0
rollup 2.46.0 2.75.5
rollup-plugin-commonjs 10.1.0 22.0.0
rollup-plugin-node-resolve 5.2.0 13.3.0
PS D:\Downloads\xeokit-convert-1.0.7> npm update npm WARN deprecated @percy/[email protected]: PercyScript has been deprecated in favor of Percy CLI's snapshot command: https://docs.percy.io/docs/percy-snapshot Please upgrade to use the CLI rather than this package, as it's no longer actively supported. npm WARN deprecated [email protected]: request-promise has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142 npm WARN deprecated [email protected]: this library is no longer supported npm WARN deprecated [email protected]: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-node-resolve. npm WARN deprecated [email protected]: jsSHA versions < 3.0.0 will no longer receive feature updates npm WARN deprecated [email protected]: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-commonjs. npm WARN deprecated [email protected]: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142 npm WARN deprecated [email protected]: support for ECMAScript is superseded by
uglify-js` as of v3.13.0
npm WARN deprecated [email protected]: This package is unmaintained and deprecated. See the GH Issue 259.
npm WARN deprecated [email protected]: Firefox support is gradually transitioning to the puppeteer package. As of puppeteer v2.1.0 you can interact with Firefox Nightly. The puppeteer-firefox package will remain available until the transition is complete, but it is no longer actively maintained. For more information visit https://wiki.mozilla.org/Remote
npm WARN deprecated [email protected]: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.
npm WARN deprecated @percy/[email protected]: @percy/agent is no longer supported. Please upgrade to @percy/cli https://docs.percy.io/docs/migrating-to-percy-cli
npm WARN deprecated [email protected]: core-js@<3.4 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
npm WARN deprecated [email protected]: core-js@<3.4 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
npm ERR! code 1
npm ERR! path D:\Downloads\xeokit-convert-1.0.7\node_modules\puppeteer-firefox
npm ERR! command failed
npm ERR! command C:\Windows\system32\cmd.exe /d /s /c node install.js
npm ERR! ERROR: Failed to download Firefox rv0.0.1!
npm ERR! Error: Download failed: server returned code 404. URL: https://github.com/puppeteer/juggler/releases/download/v0.0.1/firefox-win64.zip
npm ERR! at D:\Downloads\xeokit-convert-1.0.7\node_modules\puppeteer-firefox\lib\BrowserFetcher.js:264:21
npm ERR! at ClientRequest.requestCallback (D:\Downloads\xeokit-convert-1.0.7\node_modules\puppeteer-firefox\lib\BrowserFetcher.js:320:7)
npm ERR! at Object.onceWrapper (node:events:642:26)
npm ERR! at ClientRequest.emit (node:events:527:28)
npm ERR! at HTTPParser.parserOnIncomingClient (node:_http_client:631:27)
npm ERR! at HTTPParser.parserOnHeadersComplete (node:_http_common:128:17)
npm ERR! at TLSSocket.socketOnData (node:_http_client:494:22)
npm ERR! at TLSSocket.emit (node:events:527:28)
npm ERR! at addChunk (node:internal/streams/readable:315:12)
npm ERR! at readableAddChunk (node:internal/streams/readable:289:9)
npm ERR! -- ASYNC --
npm ERR! at BrowserFetcher. (D:\Downloads\xeokit-convert-1.0.7\node_modules\puppeteer-firefox\lib\helper.js:32:15)
npm ERR! at Object. (D:\Downloads\xeokit-convert-1.0.7\node_modules\puppeteer-firefox\install.js:47:16)
npm ERR! at Module._compile (node:internal/modules/cjs/loader:1105:14)
npm ERR! at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
npm ERR! at Module.load (node:internal/modules/cjs/loader:981:32)
npm ERR! at Function.Module._load (node:internal/modules/cjs/loader:822:12)
npm ERR! at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
npm ERR! at node:internal/main/run_main_module:17:47npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\thiago.marques\AppData\Local\npm-cache_logs\2022-06-01T17_28_25_233Z-debug-0.log`
Latest versions of the xmldom
lib are now only published as @xmldom/xmldom
, so to update from version 0.6.0 which is known to be vulnerable, it's required to switch to the scoped package. For more details see xmldom/xmldom#271
I'm one of the maintainers and I can provide an update PR if you want.
there are several errors in the file.
the function _flagSolidGeometries() runs into an error because sometimes
let maxNumPositions = -1; let maxNumIndices = -1;
are not changed and an Array with a negativ length can not be set.
in the function convert2xkt in
case "laz":
the variable rotateX is missing.
and the async function parse() can not get a loader when the data is a Buffer.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.