Comments (6)
The bottleneck seems to be writing the header key words, which I currently do one at a time.
Doing these in bulk would be better, but will take some work
from fitsio.
Can you tell if it is a fitsio or cfitsio issue?
from fitsio.
On 6/25/15, Felipe Menanteau [email protected] wrote:
Can you tell if it is a fitsio or cfitsio issue?
it seems to be cfitsio, or at least the way I'm using cfitsio
Erin Scott Sheldon
Brookhaven National Laboratory erin dot sheldon at gmail dot com
from fitsio.
I have fixed this partially. Writing the header is now much faster.
This is now in master. The overall writing is probably fast
enough to be usable, but see below
Also, the overall write speed on a normal disk is now faster than pyfits
On ext3
Using fitsio version 0.9.7
# Reading in DECam_00158995_09.fits
# Done: 0m 0.11s
# Writing file test.fits
# Done: 0m 0.33s
# ALL done: 0m 0.44s
# Reading in DECam_00158995_09.fits
Filename: DECam_00158995_09.fits
No. Name Type Cards Dimensions Format
0 SCI PrimaryHDU 360 (2048, 4096) float32
1 MSK ImageHDU 108 (2048, 4096) int16
2 WGT ImageHDU 72 (2048, 4096) float32
None
# Done: 0m 0.02s
# Writing file test.fits
# Done: 0m 1.44s
# ALL done: 0m 1.45s
On GPFS
However, the actual image write is still slower on GPFS, with pyfits
taking 2s and fitsio 30s. 30s is tolerable but we can do better. I'll
be looking into it.
from fitsio.
I have also fixed the image writing. The writing on GPFS is now also faster than pyfits
fitsio GPFS
Using fitsio version 0.9.7
# Reading in DECam_00158995_09.fits
# Done: 0m 0.15s
# Writing file test.fits
# Done: 0m 1.27s
# ALL done: 0m 1.42s
pyfits GPFS
# Reading in DECam_00158995_09.fits
Filename: DECam_00158995_09.fits
No. Name Type Cards Dimensions Format
0 SCI PrimaryHDU 360 (2048, 4096) float32
1 MSK ImageHDU 108 (2048, 4096) int16
2 WGT ImageHDU 72 (2048, 4096) float32
None
# Done: 0m 0.04s
# Writing file test.fits
# Done: 0m 2.45s
# ALL done: 0m 2.49s
from fitsio.
please pull from master and give it a try
from fitsio.
Related Issues (20)
- Return requested row order rather than sorted
- Bug when installing fitsio in one of the new Apple M2 Max HOT 18
- 1.1.9 segfaults in test_table_read_write() on i586 HOT 3
- Image return None value HOT 13
- cfitsio build failure HOT 8
- GZIP_2 does not appear to do lossless compression HOT 4
- lossless gzip not finalizing wile FITS object is open
- Please consider forwarding cfitsio adjustments to upstream HOT 2
- Consider support for `numpy` 2 variable length string `StringDType`
- move to ruff for linter in order to help flag numpy2 issues HOT 1
- Build failures against NumPy 2.0 with GCC 14 HOT 8
- 1.2.2 on pypi? HOT 4
- fitsio error while pip installing HOT 5
- bzip and curl checks don't seem to be working
- CFITSIO changed license: incompatible with GPL HOT 14
- Error when using pip install for fitsio HOT 5
- Error when install fitsio on linux, both 'pip install fitsio' and 'python setup.py install' failed. HOT 5
- Issue with slicing HOT 5
- fits_hcompress.c HOT 4
- Licensing under the LGPL
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fitsio.