tomasz-lewicki / ai-thermometer Goto Github PK
View Code? Open in Web Editor NEWFever screening with IR & RGB cameras and Deep CNNs
Fever screening with IR & RGB cameras and Deep CNNs
Hi there how can make this code run using a usb camera, instead of the camera suggested in your article.
Hello, need some advice please.. why I am getting syntax error when executing Capture Images.ipynb example?
Thank you and Regards!!!
!gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=1 ! video/x-raw,format=GRAY16_LE ! videoconvert ! pngenc ! filesink location="$save_path"
thanks for sharing us your work.
I understood that you use NVidia Nano Dev Kit with 2 cameras: Visible and Lepton 3.5.
Could you please share me information which visible camera you use and how to connect all camera with Nano? via MIPI-CS2 or SPI?
Thanks
Hi @tomek-l ! I hope you could help me.
I am a graduating college student. I have a thesis about face mask detection and body temperature detection.. I am currently working on my model for the mask detection, however I really do not know how to get started with the temperature detection. I also have the same idea which is to detect the inner eye canthus for temperature measurement, (using haar as well?). I am also planning to buy the same materials. But I really don't know how the code works or how I will calibrate the cameras. I am still a beginner in machine learning with python because I studied on my own. I hope you could help me, I really want to learn. This is the only project I found helpful. Thank you so much! 😅
need requirements.txt
hi @tomek-l
First of all, i wanna thank you for your codes. it really helps the communities at my place in this pandemic situation. however, these codes could only measure temperature with face straight to the camera and no mask used. i wonder if you have codes version 0.1 with SSD detection. I would appreciate it if you want to upload it.
Hello, I've been wondering if it is possible to access the feed the program provides(thermal and normal camera, along with the temperature overlay) so i can stream it over http. For the streaming I usually use VLC's component. My main issue is that I cannot find a way to access the feed. Is it saved locally somewhere? Am I missing something? Thanks for your time
Make sure you put a "v" in front of your torchvision export.
export BUILD_VERSION=0.x.0 # where 0.x.0 is the torchvision version
E.g. export BUILD_VERSION=v0.9.0
It always shows human body temperature around 32-34 degree Celsius but actual temperature is always around 36-37 degree. How can I get accurate values? Are you using any calibration of Lepton 3.5 camera?
Hello,
Do you know if the AI.thermometer can work in the new Jetson Nano 2Gb or I need to modify the code?
Thank you and best regards
i want to get a cv2.putext and thermal(degree) information ur file..
but i cannot find it
please help me ;)
Hello, thank you very much for your codes.
Just want to know if the face detection is possible if a person is using a mask. Thank you in advance!
Hi Tomasz,
Thank you very much for your answer concerning the script you have developped for temperature capture with Lepton 3.5.
Not being experienced in software development, I send you below a few questions:
How is captured the 16 bits flow of the Lepton camera ? Does the Python script command the camera (in line 47 to 67 or 218) ? Or do we have to send separate commands through the CCI channel ?
Which data contains the libuvc and how is it fed ?
Which command stops the flow ?
Have you an example of the temperature array (both link in your mail points to the script)
Which configuration is needed to run the script ? Is Lepton 3.5 with PureThermal2 connected via USB to a PC with windows 10 acceptable ?
Which software do we have to load (Python, openCV...) ?
Sorry to take your time by basic questions, but this could help the project a lot.
I use PI 4 and when it runs there is an error /home/pi/ai-thermometer/ir/libuvc_wrapper/libuvc.so: wrong ELF class: ELFCLASS64,how to fix? thx dude
hi @tomek-l is there a way to execute a FFC?, currently executing the code but the weather is fairly warm and is not working i need to execute an FFC.
thank you
Hello,
Thanks for all the help so far, I have an issue whereby the storage on my Jetson Nano fills up after a few hours. I have no value in saving the frames. I only use this system for a live feed on a monitor.
Is there a way I can stop it from saving frames?
Thanks
Hello, I am a student studying artificial intelligence. Thanks to the code you posted, I was able to learn a lot. Can you upload the code for "Version 0.2: Haar Cascade + FLIR Lepton 3.5 temp. measurement (~10FPS, CPU only)"? I think it will be very helpful for me who is currently studying Haar-based recognition code. I'd appreciate it if you could post it.
Thank you again for all the help!!!
Can you explain me how you fix it please? When applying the "dirty solution" I get:
bash: ENV{DEVTYPE}=="usb_device,: command not found
What I am doing wrong?
Best regards!!!
I'm leaning towards using SPI directly (rather than USB) for interfacing Lepton 3.5 in a future hardware revision.
If you're using AI Thermometer and have any thoughts on this change, feel free to comment below.
I think it’s better if we can use the InfiRay Tiny1-B Micro LWIR Thermal Module sensor instead of the FLIR LEPTON 3.5
The Tiny1-B sensor have a biological temperature measurement accuracy of ±0.5°C and the FLIR Lepton have a measurement accuracy of ±5°C
The sensor is cheap and we can avoid the use of a black box in order to constantly calibrate the sensor. It have radiometric capabilities.
In that way we can have a very useful instrument without the accuracy limitation of the FLIR sensors.
https://www.infiray.com/products/tiny1-b-micro-lwir-thermal-module.html
Regards
Hi, I'm using Lepton 3.5 and purethermal 2, but my result temperature is around 100 (it should be close to 37,right?), and when I use the notebook(https://github.com/tomek-l/ai-thermometer/blob/master/docs/notebooks/Getting%20Started%20with%20Cameras.ipynb), the raw data for each pixel is around 8000 and after Kelvin to Celsius the result is around -195.
Do you have any idea about this? Thanks.
AI Thermometer should have an option for online calibration with an external blackbody reference for more accurate readings.
My general idea to implement this is as follows:
I plan to add this in V0.5 of the software.
Any suggestions on how to realize that feature are welcome :)
I copied your code and am running it on the developer kit. Unfortunately the FLIR Lepton shows the pictures only in red and grey like in your video with the red channel. How can i change the output of the FLIR Lepton to a range from blue to red like in your example?
Hi,
Thanks so much for your work this is really interesting.
I was wondering what kind of interface have you used for actual deployment. I saw you were working on a web interface, does that work? For us to put on a screen near the camera display?
Would be nice to have like an audio alert which we could play maybe through the web interface if someone is detected above the desired range.
Also curious if you're using the flir 3.5 as well, do you need any calibration? My detection seems to be a little bit off
Thanks again
Hi,
Just wanted to propose and to see if there is anyway to improve consistency and accuracy of the face detection when people are wearing masks or when the target is moving. Maybe training the model with people wearing masks and not wearing masks.
Let me know what you think and if it is doable.
Nick
Hi there, how do I change both images like 180 degrees?
Originally posted by @jjferreira in #39
I'm getting a message Waiting for RGB frames any idea what I am missing?
Estimated / selected altsetting bandwith : 18 / 642.
INFO: Using $DISPLAY from environment, not from config
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/nano-termo/ai-thermometer/rgb/rgb_thread.py", line 38, in run
frame_small = cv2.resize(self._frame, (400, 300))
cv2.error: OpenCV(4.4.0) /tmp/build_opencv/opencv/modules/imgproc/src/resize.cpp:3929: error: (-215:Assertion failed) !ssize.empty() in function 'resize'
Has anyone run into calibration issues when using this outdoors - particularly in winter? The system works perfectly inside with ambient temperatures of 66-72F. However, trying to use it outdoors <45F produces really drastic readings on the facial detections - typically over 1000F!
I've tried leaving the system outside but am still having the same problems. I'm a track & field coach and am trying to set this up to screen athletes so I need it to work outside. Any suggestions would be greatly appreciated.
Thanks!
Hi there, how do I change both images like 180 degress?
I want to change the model used in SSD Detector with my custom trained model which is an onnx and it is an facemask detector. Is it possible to use my model to detect facemask and the temperature of the person? I have modified the detector.py in SSD folder into this to load the my model :
parent_dir_pth = os.path.dirname(os.path.abspath(file))
onnx_model_pth = (
parent_dir_pth + "/caffe/ssd-mobilenet.onnx"
)
self._net = cv2.dnn.readNetFromONNX(onnx_model_pth)
but I get this error
Loading SSD weights from file...
Traceback (most recent call last):
File "main.py", line 220, in
rgb_thread = RGBThread(model=FACE_DET_MODEL)
File "/home/g1-facemask/ai-thermometer/rgb/rgb_thread.py", line 18, in init
self._detector = SsdDetector()
File "/home/g1-facemask/ai-thermometer/rgb/ssd/detector.py", line 18, in init
self._net = cv2.dnn.readNetFromONNX(onnx_model_pth)
cv2.error: OpenCV(4.4.0) /tmp/build_opencv/opencv/modules/dnn/src/dnn.cpp:604: error: (-2:Unspecified error) Can't create layer "433" of type "Exp" in function 'getLayerInstance'
I am using OpenCV4.4.0 and latest version of pytorch
Is there a thing that I can do?
Should I retrain my model that gets caffemodel and prototxt like the default? If yes, is there any tutorial to do that kind of training? @tomek-l
does anyone have a .STEP or .STL? I need to send a file to a 3rd party to print the enclosure.
thank you!
I am implementing ai thermometer with your kind explanation.
Now I am building the environment of Raspberry pi camera v2 & lepton2.5 & PureThermal2.
But nothing has happened since Uvc_open error -3 appeared after python3 main.py.
Can I know the solution?
Of course, the individual operation of each sensor was confirmed.
thank you.
Hello
I want to work on the AI Thermometer project and I want it to work on Ubuntu system, but I do not have any of the hardware components, and if it can run on Raspberry Pi 3, I am working on it for the graduation project
could you please help me
Hello @tomek-l. First of all, brilliant work on the code.
I am using webcam and lepton 3.5 through Purethermal mini(connected using USB) on windows platform. I am just trying this code in my laptop. so, i believe there wont be a use for gstreamer. I would like to know how to use this code on windows. I tried writing my own program, but the temperature readings are not accurate. In lepton 3.5/2.5 documentation, it has been mentioned that the temperature value will be stored only when the t-linear mode is enabled. In your code, how have you done that? Also please let me know how to use your code on windows. what are the changes that I need to make? please let me know.
Also I am unable to view the image using matplotlib.
Thanks.
File "main.py", line 13, in
from rgb import RGBThread
File "/home/thermal/ai-thermometer/rgb/init.py", line 1, in
from .rgb_thread import *
File "/home/thermal/ai-thermometer/rgb/rgb_thread.py", line 8, in
from .retinaface import RetinaFaceDetector
File "/home/thermal/ai-thermometer/rgb/retinaface/init.py", line 1, in
from .detector import *
File "/home/thermal/ai-thermometer/rgb/retinaface/detector.py", line 10, in
from .models.retinaface import RetinaFace
File "/home/thermal/ai-thermometer/rgb/retinaface/models/retinaface.py", line 5, in
import torchvision.models.detection.backbone_utils as backbone_utils
ModuleNotFoundError: No module named 'torchvision.models.detection'
I am trying to use Neopixels on this project,. Recently Adafruit coded some libraries for the Neopixels that are useful in the Jetson Nano using CircuitPython. It works pretty nice...
I want to make the Neopixel stay ON with green colour when the Jetson Nano be in fever screening mode and when a 37.5 C temperature is detected, the Neopixel changes the colour to red for example... I need some help to incorporate it to the Tomasz project please.
Thank you.
Here is a simple tutorial to make Neopixels work with the Jetson Nano:
Install Pip3 if not already installed:
sudo apt-get install python3-pip
Reboot
Install Adafruit-Blinka 5.8.0:
pip3 install Adafruit-Blinka
Enable SPI on Jetson Nano:
sudo /opt/nvidia/jetson-io/jetson-io.py
Select Configure 40-pin expansion header at the bottom.
Select spi1 (19, 21, 23, 24, 26) and then select Back
Finally select Save and reboot to reconfigure pins. This will create a config file and reboot the Jetson Nano.
After the Nano boots up again, verify you have the I2C devices with the command:
ls /dev/i2c* /dev/spi*
You can test to see what I2C addresses are connected by running sudo i2cdetect -r -y 0
(on pins 27/28) or sudo i2cdetect -r -y 1
(on pins 3/5)
Then, we need to set user permissions. In order to use the Jetson GPIO Library, the correct user permissions/groups must be set first. Start by creating a new gpio user group:
sudo groupadd -f -r gpio
sudo usermod -a -G gpio
Clone the Repo and copy the rules:
cd ~
git clone https://github.com/NVIDIA/jetson-gpio.git
sudo cp ~/jetson-gpio/lib/python/Jetson/GPIO/99-gpio.rules /etc/udev/rules.d
This section was originally written for releases before JetPack 4.3. For JetPack 4.3, the udev rules are in: /lib/udev/rulesd/60-jetson-gpio-common.rules. This should mean that you don’t have to copy the file anymore with JetPack 4.3 and above.
Also, it appears as if the default is for users to be included in the gpio group. You can check this by:
$ groups
Reboot
Update just in case if a library is missing:
sudo pip3 install --upgrade adafruit_blinka
Install CircuitPython NeoPixel_SPI library;:
pip3 install adafruit-circuitpython-neopixel-spi
Reboot again and that´s all!!!
Connect "Din" of your Neopixel to the SPI1_MOSI pin of your Jetson Nano(pin 19 of the expansion header), Gnd to Gnd and 5v to 5v.
try this example:
import board
import neopixel_spi
pixels = neopixel_spi.NeoPixel_SPI(board.SPI(), 10)
pixels.fill(0xff0000)
If an error occurs
Check you have installed this dependencies:
Adafruit CircuitPython BusDevice
pip3 install adafruit-circuitpython-busdevice
Pypixelbuf
pip3 install adafruit-circuitpython-pypixelbuf
Install nano (just in case the nano editor is not installed)
sudo apt-get install nano
Hello, can you link the pre-made SD card Image?
Also what ports is the Adafruit Lightstick plugged/soldered too?
Hi,
First of all, thanks for sharing the source code for this project, very helpful. I have been trying to run this code with a web cam instead of using the raspberry pi cam. I have it working with display and face detection, the only issue I am running into is that the temperature readings are not accurate. Would you have any idea what could be causing this behavior?
I have 4.1 from jetsonhacks.
Hey Tomasz
Thanks for putting this out for people its a really cool project bro. I'm having trouble finding the video; I get the following error. Any idea trouble where I messed up? I'm thinking it may be how cameras are lined up. I 3d printed the case you gave but I noticed its slightly different than what you showed in the blog post. I attached a picture of how it look on my end was hoping if you thought that was maybe the problem. Also I 3d printed an Iron Man Helmet as a case I think you would appreciate. I attached that as well.
Thanks again,
Raza
/ai-thermometer$ python3 main.py
Loading weights from file...
Weights loaded!
Running first net inference...
Detector initialized!
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 3264 H = 2464
seconds to Run = 0
Frame Rate = 21.000000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
[ WARN:0] global /tmp/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
uvc_open error -3
I just wanna display the video of the highest temperature on IR thermal image, what file pack I need to download and which module I need to download on offcial lepton website?
I am using a simple logitech usb camera with jetson xavier but I am getting a uvc_find_error error but the camera is getting detected and activated. Everything is working well with simple codes.
Hey,
Thanks for this project, I've had great fun working on this, an area I am really new too. A quick question which may be worth updating in the notes, which version is this? It appears to be v.01 with the simple box detection? Is that correct? I'm really interested in the eye detection model 👍
Many thanks,
David
A lot of people are asking about the previous detectors (like Haar Cascades). It would be valuable if we could change different detectors in a plug-and-play fashion.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.