Comments (8)
Hi @lugfug and thank you for submitting this issue!
Now, we've encountered a similar error with Vercel some time ago - back then, it was caused by Vercel's bundler (@vercel/nft) which was dropping our ML models fingerprint-generator
was using. I've tried writing this fingerprint-generating Lambda myself (sans your helper methods) and it worked just fine (see example below).
const { FingerprintGenerator } = require('fingerprint-generator');
exports.handler = async (event, context) => {
const fp = new FingerprintGenerator();
const fingerprint = fp.getFingerprint();
return {
statusCode: 200,
body: fingerprint,
headers: {
'Content-Type': 'application/json',
},
};
};
Can you try to replace your code with this snippet and confirm whether it works in your environment?
If this doesn't help, can you please share more details about your setup? Are you using some bundlers, how do you manage dependencies in your project etc.? Thanks! :)
from fingerprint-suite.
Hello again @lugfug , I'm going to close this issue due to inactivity now - but in case you have any questions, feel free to reopen it any time.
Thanks!
from fingerprint-suite.
Hello @barjin ,
Thank you for your previous guidance. I've taken the steps you recommended and wanted to provide an update on my findings.
Please excuse the delay in my reply.
I have been working on this project and building it out over nearly 500 revisions.
I can confirm that the entire script functions exactly as expected... until I integrate the fingerprint generation code.
The short version of what the script does. A search URL is provided to the script, It then sets up a Playwright browser with generic headers (which I and currently constructing from various public lists), and then it connects via proxies to the search URL and processes the page.
I would rather use your Fingerprint-Suite to generate the headers all in one spot, vs. my current version of building the headers from multiple processes.
1. Code Update
Based on your suggestion, I integrated the FingerprintGenerator
instantiation and fingerprint generation into my AWS Lambda function. Here's the code I used at the very top of my script before any other code had a chance to run.:
const { FingerprintGenerator } = require('fingerprint-generator');
const version = require('./package.json').version;
exports.handler = async (event, context) => {
// Log the version number of the script
console.log(`Script Version: ${version}`);
// Log all the parameters passed to the script
console.log(`Received event: ${JSON.stringify(event)}`);
try {
if (event.generate_fingerprint) {
const fp = new FingerprintGenerator();
const fingerprint = fp.getFingerprint();
// Log the generated fingerprint
console.log(`Generated Fingerprint: ${fingerprint}`);
return {
statusCode: 200,
body: fingerprint,
headers: {
'Content-Type': 'application/json',
},
};
}
} catch (error) {
console.error("Error occurred:", error.message);
console.error("Error trace:", error.stack);
throw error; // re-throw the error to ensure it's not silently handled
}
};
2. Logs from AWS Lambda Execution
Upon executing the above code, here's the log output I received:
INIT_START Runtime Version: nodejs:18.v12 Runtime Version ARN: arn:aws:lambda:us-west-1::runtime:0bdff101a7b4e0589af824f244deb932XXXXXXXXXXXXXXXXXXXXXXX
START RequestId: e32a251a-7ffc-462XXXXXXXXXXXXXXXXXXXXXX Version: $LATEST
2023-09-09T21:27:50.085Z e32a251a-7ffc-4622-ae20-ba9582a766dd INFO Script Version: 0.0.478
2023-09-09T23:24:24.289Z 33829e2f-3e28-4e91-b805-7891484500dc ERROR Invoke Error
{
"errorType": "Error",
"errorMessage": "Invalid filename",
"stack": [
"Error: Invalid filename",
" at new module.exports (/opt/nodejs/node_modules/adm-zip/adm-zip.js:57:19)",
" at new BayesianNetwork (/opt/nodejs/node_modules/generative-bayesian-network/bayesian-network.js:24:21)",
" at new HeaderGenerator (/opt/nodejs/node_modules/header-generator/header-generator.js:89:38)",
" at new FingerprintGenerator (/opt/nodejs/node_modules/fingerprint-generator/fingerprint-generator.js:15:9)",
" at exports.handler (/var/task/index.js:58:24)",
" at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1147:29)"
]
}
END RequestId: e32a251a-7ffc-4622-ae20-ba9582a766dd
REPORT RequestId: e32a251a-7ffc-4622-ae20-ba9582a766dd Duration: 9.78 ms Billed Duration: 10 ms Memory Size: 3000 MB Max Memory Used: 124 MB Init Duration: 1149.48 ms
It seems the "Invalid filename" error is still persisting and is originating from the adm-zip
module used by generative-bayesian-network
, and subsequently, the fingerprint-generator
.
3. Additional Details
I've ensured that:
- My AWS Lambda configuration is correct.
- The contents of my deployment package are complete and correctly named.
- I'm using the following module versions:
"adm-zip": "^0.5.10",
"aws-sdk": "^2.1438.0",
"axios": "^1.4.0",
"cheerio": "^1.0.0-rc.12",
"fingerprint-generator": "^2.1.39",
"fingerprint-injector": "^2.1.39",
"generative-bayesian-network": "^2.1.39",
"header-generator": "^2.1.39",
"https-proxy-agent": "^7.0.1",
"playwright-aws-lambda": "^0.10.0",
"playwright-core": "^1.37.0",
"top-user-agents": "^1.0.60",
"unique-random-array": "^3.0.0"
If you need more information about my environment I'll be happy to provide it.
Given the continuation of this issue, even after trying your proposed solution, I would truly appreciate any further insights or recommendations you might have. Thank you for your continued assistance.
Warm regards,
[lugfug]
from fingerprint-suite.
Hi, I've closed the new issue and reopened this one, so all the context is in one thread.
Unfortunately(?), I just ran your code example without any issues on AWS. I've attached a .zip
archive I used to deploy it on AWS. Try creating a whole new Lambda (with all the basic / default permission settings) and upload code as zip (and use the zip archive below).
It's definitely an interesting issue - all the more so because I cannot reproduce it. Definitely keep me updated on whether you've managed to run this zip at least. Thanks!
from fingerprint-suite.
@barjin Thank You for re-opening this issue. I will create a new test AWS Lambda Function and try your package.
Theoretically, there must be a permissions issue that is causing the issue on my end.
FYI, I have all my node modules in an AWS Lambda Layer for redundancy, and reusability across Lambda Functions.
Likewise using a Lambda Layer allows the Lambda Function to have a very small foot print when uploading the deployment package.
This strategy avoids the deployment package size limitation.
from fingerprint-suite.
Hello @barjin,
I've pinpointed the underlying issue.
Firstly, I deployed your code and modules into a new AWS Lambda Function. The outcome was positive: your code executed successfully, both with and without implementing a Lambda Layer.
Subsequently, I swapped out your index.js for my existing Lambda Function's index.js, which led to a failed function invocation.
Upon comparing your modules folder with mine, I noticed a significant size difference. This was observed only for the modules that both of us had in common.
Afterward, I replaced all my corresponding modules with yours in my Lambda Layer deployment. On invoking the Lambda Function again, everything worked perfectly! The Fingerprint Suite code executed as intended, generated a fingerprint, and logged the results as JSON to the console.
I've recreated this scenario multiple times using various permutations, and I can consistently reproduce either the failure or success based on the modules used.
This investigation strongly suggests a dependency conflict as the root cause.
For further clarity, I'll attach two zip files: one containing your modules that work seamlessly, and the other with my modules that consistently fail upon deployment.
node_modules_barjin.zip
node_modules_lugfug.zip
What perplexes and concerns me is that I had installed the npm packages individually, specifically to sidestep such dependency conflicts.
@barjin, could you clarify the recommended method for both installing and updating the fingerprint-suite? Additionally, it might benefit everyone if these instructions were further explicitly detailed in the README.md.
from fingerprint-suite.
Well, looking into your node_modules_lugfug.zip
file, I see that your version is missing the contents of the (header|fingerprint)-generator/data_files
folder. The error message you see on AWS is absolutely right, i.e. adm-zip
cannot find the files (because they do not exist).
I doubt this is because of dependency conflicts, I'd wager that it's caused by your tooling (perhaps your zip
implementation omits zip files from the contents of the file getting currently zipped?)
Either way, the only correct way of installing and updating the fingeprint-suite
packages is npm install
and npm update
, which should install the packages correctly, i.e. with the data .zip
files.
from fingerprint-suite.
Hi @barjin,
Thank you for your assistance throughout this process. After a thorough investigation, we've identified the root cause of the issue. It was indeed related to the zip command used for deployment. The command was set to exclude .zip
files, which led to some necessary files being omitted from the node_modules
directory in the AWS Lambda Layer deployment zip file. This caused the AWS Lambda function to fail upon invocation due to missing dependencies.
I've resolved the issue by modifying the deployment command to allow .zip
files in the node_modules
directory while still excluding .zip
files in other root directories.
This case highlights the importance of carefully configuring deployment commands to ensure all necessary files are included in the final package. I'll make sure to thoroughly test the function after deployment in the future to confirm it's working as expected.
Thank you again for your help. I'm closing this support request now, but I'll reach out if we encounter any further issues.
Best regards,
@lugfug
from fingerprint-suite.
Related Issues (20)
- Opera User Agent Missing. HOT 1
- Prompt browser is not safe, how to solve this problem? HOT 1
- Invalid filename after Electron build HOT 1
- Random useragent HOT 1
- Bug in platform generation `"platform": "Win32"` HOT 3
- Invalid Filename Error on Instantiating FingerprintGenerator in AWS Lambda
- Antibot system HOT 1
- Relaxing the constraints & Strict mode HOT 1
- Fingerprint of my browser to fingerprint-suite HOT 1
- Nowsecure.nl does not pass HOT 4
- Bug fonts in fingerprint generator HOT 1
- navigator.userAgentData.getHighEntropyValues() doesn't return the values of base hints HOT 1
- Browser hangs when navigating on shopee.com.my when overriding codecs HOT 4
- Please add documentation
- navigator.mediaDevices maybe nullable HOT 1
- What are all the "fingerprintOptions" options of newInjectedPage() from "fingerprint-injector" HOT 1
- `sec-ch-ua` header is weirdly inconsistent HOT 4
- No Docs/API reference? HOT 2
- Chrome WebView User-Agent HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fingerprint-suite.