zdavatz / aips2sqlite Goto Github PK

aips2sqlite - create an SQLite DB from AIPS, Swissmedic, BAG and Refdata

CSS 0.78% Java 99.20% Makefile 0.02%

aips2sqlite sqlite java aips swissmedic bag refdata

aips2sqlite's Issues

Unmarshalling Swissmedic xml... Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

I do java -jar -Xmx6048m aips2sqlite.jar --lang=de --xml
This results in:

Unmarshalling Swissmedic xml... Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOfRange(Arrays.java:3664)
        at java.lang.String.<init>(String.java:207)
        at java.lang.StringBuilder.toString(StringBuilder.java:407)
        at com.sun.xml.internal.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl$StringImplImpl.parse(RuntimeBuiltinLeafInfoImpl.java:1022)
        at com.sun.xml.internal.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl$StringImplImpl.parse(RuntimeBuiltinLeafInfoImpl.java:1015)
        at com.sun.xml.internal.bind.v2.runtime.reflect.TransducedAccessor$CompositeTransducedAccessorImpl.parse(TransducedAccessor.java:230)
        at com.sun.xml.internal.bind.v2.runtime.unmarshaller.LeafPropertyLoader.text(LeafPropertyLoader.java:50)
        at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.text(UnmarshallingContext.java:572)
        at com.sun.xml.internal.bind.v2.runtime.unmarshaller.SAXConnector.processText(SAXConnector.java:198)
        at com.sun.xml.internal.bind.v2.runtime.unmarshaller.SAXConnector.endElement(SAXConnector.java:161)
        at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
        at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source)
        at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
        at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
        at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
        at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
        at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:243)
        at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:214)
        at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:157)
        at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:204)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.readAipsFile(Aips2Sqlite.java:432)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.generateSQLiteDB(Aips2Sqlite.java:351)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:334)

openjdk 17.0.5 build error

Install openjdk-17.0.5
cd aips2sqlite
run make aips2sqlite
results in

./gradlew jar

> Task :compileJava FAILED
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/MedicalInformations.java:31: error: cannot access jakarta.xml.bind.annotation.XmlAccessType
import jakarta.xml.bind.annotation.XmlAccessType;
                                  ^
  bad class file: /home/zeno/.gradle/caches/modules-2/files-2.1/jakarta.xml.bind/jakarta.xml.bind-api/4.0.0/bbb399208d288b15ec101fa4fcfc4bd77cedc97a/jakarta.xml.bind-api-4.0.0.jar(jakarta/xml/bind/annotation/XmlAccessType.class)
    class file has wrong version 55.0, should be 52.0
    Please remove or make sure it appears in the correct subdirectory of the classpath.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s
1 actionable task: 1 executed
make: *** [Makefile:4: aips2sqlite] Fehler 1

Refdata Partner File download not always working

Running java -jar -Xmx7048m aips2sqlite.jar --lang=fr --xml
Results in Refdata Partner File not being able to be downloaded, or taking a very long time:

- Downloading Refdata partner file... -Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.base/java.util.Arrays.copyOf(Arrays.java:3537)
        at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:228)
        at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:682)
        at java.base/java.lang.StringBuilder.append(StringBuilder.java:218)
        at java.base/java.util.regex.Matcher.appendReplacement(Matcher.java:999)
        at java.base/java.util.regex.Matcher.replaceAll(Matcher.java:1181)
        at java.base/java.lang.String.replaceAll(String.java:2939)
        at com.maxl.java.aips2sqlite.AllDown.downRefdataPartnerXml(AllDown.java:355)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:394)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)

Some times the second attempt it seems to work again, but not always.

Swissmedic hat einen neuen Download Link

https://www.swissmedic.ch/dam/swissmedic/de/dokumente/internetlisten/zugelassene_packungen_ham.xlsx.download.xlsx/Zugelassene_Packungen%20HAM_31012019.xlsx

Cannot get a text value from a numeric cell

Running this command: java -jar -Xmx7048m build/libs/aips2sqlite.jar --lang=fr --xml
There seems to be an issue with SwissmedicPackungen.xlsx

>> ERROR: 4460 - SwissmedicNo5 not found in Swissmedic Packungen.xls - (62257) Zolédronate Axapharm Ostéo 5, Solution pour perfusion
Exception in thread "main" java.lang.IllegalStateException: Cannot get a text value from a numeric cell
        at org.apache.poi.xssf.usermodel.XSSFCell.typeMismatch(XSSFCell.java:845)
        at org.apache.poi.xssf.usermodel.XSSFCell.getRichStringCellValue(XSSFCell.java:294)
        at org.apache.poi.xssf.usermodel.XSSFCell.getStringCellValue(XSSFCell.java:246)
        at com.maxl.java.aips2sqlite.BaseDataParser.parseSwissmedicPackagesFile_Gtin(BaseDataParser.java:167)
        at com.maxl.java.aips2sqlite.SwissMedSequences.process(SwissMedSequences.java:180)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:271)

Heap space error

Running this java -jar -Xmx7048m build/libs/aips2sqlite.jar --lang=fr --xml
I get

Axapharm Osteo 5, Infusionslösung
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
   at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300)
   at java.lang.StringCoding.encode(StringCoding.java:344)
   at java.lang.String.getBytes(String.java:918)
   at com.maxl.java.aips2sqlite.HtmlUtils.calcHashCode(HtmlUtils.java:1028)
   at com.maxl.java.aips2sqlite.HtmlUtils.addHeaderToXml(HtmlUtils.java:990)
   at com.maxl.java.aips2sqlite.RealExpertInfo.process(RealExpertInfo.java:1304)
   at com.maxl.java.aips2sqlite.Aips2Sqlite.generateSQLiteDB(Aips2Sqlite.java:356)
   at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:334)

„letzte Bestelldatum“ auch im JSON auslesen

Nota.csv hat zuhinterst ein neues Feld: letzte Bestelldatum, das muss im SO json auch ausgelesen werden.

generate JSON files along ser files

The easy way is to make a temporary modification to the Java code to generate the JSON file as well as the Serialized file, then we know what to put in there.

for

rose_autogenerika.ser.clear
rose_ids.ser.clear
rose_conditions.ser.clear
rose_nota.ser.clear

Néo -> Neo

Leider lässt sich in der Datenbank das Néo Mercazole nicht direkt finden, da es mit Néo geschrieben ist, über den Wirkstoff geht es dann…

make aips2sqlite fails with

I am trying this on my Linux machine.

If I do "make aips2sqlite" it fails with:

~/.software/aips2sqlite> make aips2sqlite
./gradlew jar

> Task :compileJava FAILED
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/BarCode.java:34: warning: BASE64Encoder is internal proprietary API and may be removed in a future release
import sun.misc.BASE64Encoder;
               ^
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/BarCode.java:58: warning: BASE64Encoder is internal proprietary API and may be removed in a future release
            BASE64Encoder encoder = new BASE64Encoder();
            ^
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/BarCode.java:58: warning: BASE64Encoder is internal proprietary API and may be removed in a future release
            BASE64Encoder encoder = new BASE64Encoder();
                                        ^
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/FileOps.java:154: error: cannot find symbol
                        Crypto crypto = new Crypto();
                        ^
  symbol:   class Crypto
  location: class FileOps
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/FileOps.java:154: error: cannot find symbol
                        Crypto crypto = new Crypto();
                                            ^
  symbol:   class Crypto
  location: class FileOps
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/FileOps.java:281: error: cannot find symbol
                        Crypto crypto = new Crypto();
                        ^
  symbol:   class Crypto
  location: class FileOps
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/FileOps.java:281: error: cannot find symbol
                        Crypto crypto = new Crypto();
                                            ^
  symbol:   class Crypto
  location: class FileOps
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/GlnCodes.java:501: error: cannot find symbol
                Crypto crypto = new Crypto();
                ^
  symbol:   class Crypto
  location: class GlnCodes
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/GlnCodes.java:501: error: cannot find symbol
                Crypto crypto = new Crypto();
                                    ^
  symbol:   class Crypto
  location: class GlnCodes
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/ShoppingCartDesitin.java:83: error: cannot find symbol
                Crypto crypto = new Crypto();
                ^
  symbol:   class Crypto
  location: class ShoppingCartDesitin
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/ShoppingCartDesitin.java:83: error: cannot find symbol
                Crypto crypto = new Crypto();
                                    ^
  symbol:   class Crypto
  location: class ShoppingCartDesitin
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/ShoppingCartIbsa.java:502: error: cannot find symbol
                Crypto crypto = new Crypto();
                ^
  symbol:   class Crypto
  location: class ShoppingCartIbsa
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/ShoppingCartIbsa.java:502: error: cannot find symbol
                Crypto crypto = new Crypto();
                                    ^
  symbol:   class Crypto
  location: class ShoppingCartIbsa
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/ShoppingCartRose.java:41: error: cannot find symbol
                Crypto crypto = new Crypto();
                ^
  symbol:   class Crypto
  location: class ShoppingCartRose
/home/zeno/.software/aips2sqlite/src/com/maxl/java/aips2sqlite/ShoppingCartRose.java:41: error: cannot find symbol
                Crypto crypto = new Crypto();
                                    ^
  symbol:   class Crypto
  location: class ShoppingCartRose
12 errors
3 warnings

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s
1 actionable task: 1 executed
make: *** [Makefile:4: aips2sqlite] Fehler 1

Update MedRegBM Links

The following Links have changed:
https://www.medregbm.admin.ch/Publikation/CreateExcelListMedizinalPersons
https://www.medregbm.admin.ch/Publikation/CreateExcelListBetriebs
They now should be
https://www.swissmedic.ch/dam/swissmedic/de/dokumente/internetlisten/medizinalpersonen_betreg.xlsx.download.xlsx/Medizinalpersonen%20BetReg%202022-10-17.xlsx
https://www.swissmedic.ch/dam/swissmedic/de/dokumente/internetlisten/betriebe_betreg.xlsx.download.xlsx/Betriebe%20BetReg%202022-10-17.xlsx
In the future we need to get the Files via API:
Factsheet_Standardschnittstellen_BetReg_3.0_DE.pdf

Remove medreg Link from Download

Remove https://www.swissmedic.ch/dam/swissmedic/de/dokumente/internetlisten/medizinalpersonen_betreg.xlsx.download.xlsx/Medizinalpersonen%20BetReg%202022-10-17.xlsx from Download.

Downloading Refdata Pharma File not working

runniung this command java -jar -Xmx7048m jars/aips2sqlite.jar --lang=fr --xml
Results in

- Downloading Refdatabase pharma file... |Nov. 15, 2022 1:33:52 PM com.sun.xml.messaging.saaj.soap.MessageImpl saveChanges
SCHWERWIEGEND: SAAJ0539: Header Stream in saveChanges kann nicht abgerufen werden
Nov. 15, 2022 1:33:52 PM com.sun.xml.messaging.saaj.soap.MessageImpl saveChanges
SCHWERWIEGEND: SAAJ0540: Fehler beim Speichern einer Multipart-Nachricht
 Exception: in 'downRefdataPharmaXml'
com.sun.xml.messaging.saaj.SOAPExceptionImpl: Error during saving a multipart message
        at com.sun.xml.messaging.saaj.soap.MessageImpl.saveChanges(MessageImpl.java:1284)
        at com.maxl.java.aips2sqlite.AllDown.downRefdataPharmaXml(AllDown.java:284)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:393)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)
Caused by: com.sun.xml.messaging.saaj.SOAPExceptionImpl: Unable to get header stream in saveChanges: 
        at com.sun.xml.messaging.saaj.soap.MessageImpl.saveChanges(MessageImpl.java:1254)
        ... 3 more
Caused by: java.io.IOException: Nicht unterstützt: http://javax.xml.XMLConstants/property/accessExternalDTD
        at com.sun.xml.messaging.saaj.soap.impl.EnvelopeImpl.output(EnvelopeImpl.java:291)
        at com.sun.xml.messaging.saaj.soap.impl.EnvelopeImpl.output(EnvelopeImpl.java:303)
        at com.sun.xml.messaging.saaj.soap.SOAPPartImpl.getContentAsStream(SOAPPartImpl.java:342)
        at com.sun.xml.messaging.saaj.soap.MessageImpl.getHeaderBytes(MessageImpl.java:1094)
        at com.sun.xml.messaging.saaj.soap.MessageImpl.saveChanges(MessageImpl.java:1246)
        ... 3 more

CAUSE:

com.sun.xml.messaging.saaj.SOAPExceptionImpl: Unable to get header stream in saveChanges: 
        at com.sun.xml.messaging.saaj.soap.MessageImpl.saveChanges(MessageImpl.java:1254)
        at com.maxl.java.aips2sqlite.AllDown.downRefdataPharmaXml(AllDown.java:284)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:393)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)
Caused by: java.io.IOException: Nicht unterstützt: http://javax.xml.XMLConstants/property/accessExternalDTD
        at com.sun.xml.messaging.saaj.soap.impl.EnvelopeImpl.output(EnvelopeImpl.java:291)
        at com.sun.xml.messaging.saaj.soap.impl.EnvelopeImpl.output(EnvelopeImpl.java:303)
        at com.sun.xml.messaging.saaj.soap.SOAPPartImpl.getContentAsStream(SOAPPartImpl.java:342)
        at com.sun.xml.messaging.saaj.soap.MessageImpl.getHeaderBytes(MessageImpl.java:1094)
        at com.sun.xml.messaging.saaj.soap.MessageImpl.saveChanges(MessageImpl.java:1246)
        ... 3 more

Full XML files `fi_de.xml` and `fi_fr.xml` are not created anymore

The full XML files fi_de.xml and fi_fr.xml are not created anymore.
The following files should be created
fi_de.xml
fi_fr.xml
These two files are currently empty but they should contain all the Fachinfos in one file.

Folder `fi_it_html` is not created

with the command java -jar -Xmx8000m jars/aips2sqlite.jar --owner=Axapharm --plain --nodown --lang=it
the folder fi_it_html is not created

Parse new file Kunden_alle_NEU.csv

Create a JSON file for Kunden_alle_NEU.csv.

Columns C-F is either 0 or 1.

0=Keine Präferenz
1=Kundengruppe (any)

outputted file will be rose_conditions_new.json

siehe auch: zdavatz/smart-order#9

main" java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1

I am running:
java -jar -Xmx7048m aips2sqlite.jar --lang=de --xml
This results in

.
.
6 - 3TC®: 53662, 53663
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
        at java.base/java.util.Arrays$ArrayList.get(Arrays.java:4165)
        at com.maxl.java.aips2sqlite.RealExpertInfo.updateSectionPackungen(RealExpertInfo.java:1548)
        at com.maxl.java.aips2sqlite.RealExpertInfo.process(RealExpertInfo.java:1245)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.generateSQLiteDB(Aips2Sqlite.java:356)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:334)

Java Version: openjdk 17.0.2 2022-01-18
If an exception occurs, the software should continue. Currently it stops after the error.

Download of Refdata.xml not longer working because of https

See: zdavatz/cpp2sqlite#203
Currently we get this error:

rm: remove regular file ‘oddb2xml_swissmedic_sequences.csv’? Feb 04, 2022 2:00:26 AM com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection post
SEVERE: SAAJ0008: Bad Response; Moved Temporarily
 Exception: in 'downRefdataPharmaXml'
com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Bad response: (302Moved Temporarily
        at com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.call(HttpSOAPConnection.java:149)
        at com.maxl.java.aips2sqlite.AllDown.downRefdataPharmaXml(AllDown.java:292)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:393)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)
Caused by: com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Bad response: (302Moved Temporarily
        at [com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.post](http://com.sun.xml.internal.messaging.saaj.client.p2p.httpsoapconnection.post/)(HttpSOAPConnection.java:264)
        at com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.call(HttpSOAPConnection.java:145)
        ... 3 more

CAUSE:

com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Bad response: (302Moved Temporarily
        at [com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.post](http://com.sun.xml.internal.messaging.saaj.client.p2p.httpsoapconnection.post/)(HttpSOAPConnection.java:264)
        at com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.call(HttpSOAPConnection.java:145)
        at com.maxl.java.aips2sqlite.AllDown.downRefdataPharmaXml(AllDown.java:292)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:393)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)

CAUSE:

com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Bad response: (302Moved Temporarily
        at [com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.post](http://com.sun.xml.internal.messaging.saaj.client.p2p.httpsoapconnection.post/)(HttpSOAPConnection.java:264)
        at com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.call(HttpSOAPConnection.java:145)
        at com.maxl.java.aips2sqlite.AllDown.downRefdataPharmaXml(AllDown.java:292)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:393)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)
Feb 04, 2022 2:00:26 AM com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection post
SEVERE: SAAJ0008: Bad Response; Moved Temporarily
 Exception: in 'downRefdataPartnerXml'
com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Bad response: (302Moved Temporarily
        at com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.call(HttpSOAPConnection.java:149)
        at com.maxl.java.aips2sqlite.AllDown.downRefdataPartnerXml(AllDown.java:354)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:394)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)
Caused by: com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Bad response: (302Moved Temporarily
        at [com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.post](http://com.sun.xml.internal.messaging.saaj.client.p2p.httpsoapconnection.post/)(HttpSOAPConnection.java:264)
        at com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.call(HttpSOAPConnection.java:145)
        ... 3 more

CAUSE:

com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Bad response: (302Moved Temporarily
        at [com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.post](http://com.sun.xml.internal.messaging.saaj.client.p2p.httpsoapconnection.post/)(HttpSOAPConnection.java:264)
        at com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.call(HttpSOAPConnection.java:145)
        at com.maxl.java.aips2sqlite.AllDown.downRefdataPartnerXml(AllDown.java:354)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:394)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)

CAUSE:

com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Bad response: (302Moved Temporarily
        at [com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.post](http://com.sun.xml.internal.messaging.saaj.client.p2p.httpsoapconnection.post/)(HttpSOAPConnection.java:264)
        at com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.call(HttpSOAPConnection.java:145)
        at com.maxl.java.aips2sqlite.AllDown.downRefdataPartnerXml(AllDown.java:354)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:394)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)
 Exception: in 'downGLNCodesXlsx'
java.io.IOException: Server returned HTTP response code: 500 for URL: https://www.medregbm.admin.ch/Publikation/CreateExcelListMedizinalPersons
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1897)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1495)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268)
        at [org.apache.commons.io](http://org.apache.commons.io/).FileUtils.copyURLToFile(FileUtils.java:1489)
        at com.maxl.java.aips2sqlite.AllDown.downGLNCodesXlsx(AllDown.java:598)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:403)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)

When running java -jar -Xmx6048m aips2sqlite.jar --lang=fr --xml

Invalid Parameters for `refdata_pharma_xml.xml`

Running java -jar -Xmx6048m aips2sqlite.jar --lang=fr --xml
Results in following two empty files
Downloading Refdata pharma file... 0 kB in 0.718 sec refdata_pharma_xml.xml.txt
Downloading Refdata partner file... 0 kB in 0.018 sec: refdata_partner_xml.xml.txt
The error seems to be Invalid Parameters.

HIN Proxy problems with Java certificate

aips2sqlite could not download anyhting because of the new HIN proxy.
Solved with
keytool -importcert -file /opt/ywesee/ProxyoutCA.pem -keystore cacerts -alias proxyout
This will be persistent, till a new Java version is installed.

Generate Fachinfo & Patinfo in Italian

Is there a plan to extract / generate also available data in Italian?
As I see JAVA files are binary data, right?

Bests

Philippe

Is the download for `Preparations.xml` working properly?

Running the following command: java -jar -Xmx7048m build/libs/aips2sqlite.jar --lang=fr --xml
results in

Downloading Preparations file... | Exception: in 'downPreparationsXml'
java.io.FileNotFoundException: Source './downloads/tmp/unzipped_tmp/Preparations.xml' does not exist
        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1074)
        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1038)
        at com.maxl.java.aips2sqlite.AllDown.downPreparationsXml(AllDown.java:404)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:395)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)

The following Error can currently be ignored, as Swissmedic is changing the download for this file.

- Downloading GLN codes files (Personen + Betriebe)... / Exception: in 'downGLNCodesXlsx'
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
        at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2023)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1125)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
        at org.apache.commons.io.FileUtils.copyURLToFile(FileUtils.java:1489)
        at com.maxl.java.aips2sqlite.AllDown.downGLNCodesXlsx(AllDown.java:599)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.allDown(Aips2Sqlite.java:403)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:226)

add missing rose_direct_subst.ser.clear as json

this file is still missing as json it seems:
rose_direct_subst.ser.clear

Consistent HTML File names for FI and PI

Bsp.:

Acido_X_pi_de -> für Deutsche Version
Acido_X_pi_fr -> für Französische Version
Acido_X_pi_it -> für Italienische Version

Die Dateien sollten immer einheitlich benannt werden für eine korrekte Integration.

Cmax is displayed wrong

Mir ist aufgefallen, dass die Bezeichnung Cmax bei Amiko nicht korrekt dargestellt wird (max verschwindet):
Bei Swissmedic wird das korrekt dargestellt:

ATC-Code von Isuprel ist falsch.

ATC-Code gemäss Fachinfo ist: C01CA02

generate aips2sqlite.jar only with command line

At the moment the aips2sqlite.jar is created using the IntelliJ GUI.
The .jar file has many dependencies, see /lib dir. These files all have to go into the final aips2sqlite.jar file.
Would be great to create the aips2sqlite.jar (currently located in jars Dir) using gradle and/or a build.xml file with AndroidStudio.
Build command should be make aips2sqlite.

fulldb Error in processCsv on row 1

running on Production

java -jar -Xmx6000m aips2sqlite.jar --lang=de --verbose --zurrose=fulldb

results in

Saved encrypted file rose_access.ami.ser
Processing BAG preparations file... 6010
>> Error in processCsv on row 1

Vollstamm_Galenic_Form_Mapping_by_Code.txt

Galenischen Mapping Code für SO einführen. Das File auf dem FTP heisst: Vollstamm_Galenic_Form_Mapping_by_Code.txt

Enable `--lang=it` for Fachinfo

This seems to work for PI:
java -jar -Xmx8000m jars/aips2sqlite.jar --owner=Axapharm --plain --nodown --xml --lang=it --pinfo
It does not work for FI, so we need to enable --lang=it for the FI to only generate the Italian FIs.
So we need to make this work:
java -jar -Xmx8000m jars/aips2sqlite.jar --owner=Axapharm --plain --nodown --xml --lang=it

do not use --nodown Option in production.

The option --nodown is only used for testing. In production do not use the option --nodown. My full path and options are:
/usr/local/src/jre1.8.0_101/bin/java -jar -Xmx6000m /home/zeno/.software/aips2sqlite/jars/aips2sqlite.jar --owner=Axapharm --plain --xml --lang=de --pinfo

Replace download URLs

Replace http://download.epha.ch/data/matrix/matrix.csv with https://raw.githubusercontent.com/zdavatz/oddb2xml_files/master/interactions_de_utf8.csv
Replace http://download.epha.ch/data/atc/atc.csv with https://raw.githubusercontent.com/zdavatz/cpp2sqlite/master/input/atc_codes_multi_lingual.txt and adapt the code to use the new file.
Replace http://download.epha.ch/cleaned/produkte.json download URL. See commit 953df2c

Dateien beim Download nicht ins Memory laden

Es sieht so aus als würde die Dateien beim Download schon ins Memory geladen werden.
Der Befehl java -jar -Xmx8000m aips2sqlite.jar --owner=Axapharm --plain --xml --lang=de --pinfo lädt zurest alle Dateien runter.
Können wir verhindern, dass schon beim Download die Dateien in das Memory (RAM) geladen werden? Die Dateien müssen wirklich nur runtergeladen werden.
Insbesondere der Download der Datei Refdata partner file dauert immer sehr lange.
Beim processing werden die Dateien dann in den Speicher geladen.
Oder wir machen eine Option --skip-partner-file

generating PIs wirft einen Error

java -jar -Xmx8000m jars/aips2sqlite.jar --owner=Axapharm --xml --lang=de --pinfo
wirft folgenden Error:

Processing Patient Infos...
40 - Acido-X: 61354
115 - Alfuzosin Axapharm: 59415
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
        at java.base/java.util.Arrays$ArrayList.get(Arrays.java:4165)
        at com.maxl.java.aips2sqlite.RealPatientInfo.updateSectionPackungen(RealPatientInfo.java:668)
        at com.maxl.java.aips2sqlite.RealPatientInfo.process(RealPatientInfo.java:569)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.generateSQLiteDB(Aips2Sqlite.java:360)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:334)

Failed info generation

When running the aips2sqlite.jar we get the following exception.

java -jar -Xmx6000m /opt/aips2sqlite-master/jars/aips2sqlite.jar --owner=******** --plain --xml --lang=de --pinfo

- Downloading AIPS file... 1154469 kB in 54.709 sec
- Downloading Packungen file... 0 kB in 0.54 sec
- Downloading Refdata pharma file... 18874 kB in 5.468 sec
- Downloading Refdata partner file... 108132 kB in 50.124 sec
- Downloading Preparations file... 42038 kB in 1.19 sec
- Downloading Swiss DRG (DE) file... 285 kB in 0.254 sec
- Downloading Swiss DRG (FR) file... 282 kB in 0.332 sec
- Downloading EPha interactions (DE) file... 10374 kB in 0.227 sec
- Downloading EPha interactions (FR) file... 10374 kB in 0.101 sec
- Downloading EPha products (DE) file... 853 kB in 0.059 sec
- Downloading EPha products (FR) file... 853 kB in 0.019 sec
- Downloading EPha ATC codes file... 366 kB in 0.059 sec
- Downloading GLN codes file (people/personen)... 2545 kB in 13.92 sec
- Downloading GLN codes file (companies/betriebe)... 250 kB in 1.566 sec

- Generating sqlite database...
- Unmarshalling Swissmedic xml... 22072 medis in 15.037 sec
- Processing packages xlsx... Exception in thread "main" org.apache.poi.POIXMLException: org.apache.poi.openxml4j.exceptions.InvalidFormatException: Package should contain a content type part [M1.13]
	at org.apache.poi.util.PackageHelper.open(PackageHelper.java:41)
	at org.apache.poi.xssf.usermodel.XSSFWorkbook.<init>(XSSFWorkbook.java:204)
	at com.maxl.java.aips2sqlite.RealPatientInfo.extractPackageInfo(RealPatientInfo.java:101)
	at com.maxl.java.aips2sqlite.RealPatientInfo.process(RealPatientInfo.java:501)
	at com.maxl.java.aips2sqlite.Aips2Sqlite.generateSQLiteDB(Aips2Sqlite.java:360)
	at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:334)
Caused by: org.apache.poi.openxml4j.exceptions.InvalidFormatException: Package should contain a content type part [M1.13]
	at org.apache.poi.openxml4j.opc.ZipPackage.getPartsImpl(ZipPackage.java:178)
	at org.apache.poi.openxml4j.opc.OPCPackage.getParts(OPCPackage.java:662)
	at org.apache.poi.openxml4j.opc.OPCPackage.open(OPCPackage.java:269)
	at org.apache.poi.util.PackageHelper.open(PackageHelper.java:39)
	... 5 more

aspectra "make aips2sqlite" fails

Set path in .bash_profile to export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64
cd aips2sqlite
type make aips2sqlite
Fails with

./gradlew jar 
Downloading https://services.gradle.org/distributions/gradle-5.0-bin.zip

Exception in thread "main" java.net.ConnectException: Connection timed out (Connection timed out)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:666)
        at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
        at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
        at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
        at org.gradle.wrapper.Download.downloadInternal(Download.java:66)
        at org.gradle.wrapper.Download.download(Download.java:51)
        at org.gradle.wrapper.Install$1.call(Install.java:62)
        at org.gradle.wrapper.Install$1.call(Install.java:48)
        at org.gradle.wrapper.ExclusiveFileAccessManager.access(ExclusiveFileAccessManager.java:69)
        at org.gradle.wrapper.Install.createDist(Install.java:48)
        at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:107)
        at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:61)
make: *** [aips2sqlite] Error 1

Looks like a Download restriction.

add Biotechnologika to full_db and atc_db

append Biotechnologika Flag from (Column E) from Swissmedic.xlsx to full_db and atc_db in column flags.

Flag name has to be BioT

siehe auch: zdavatz/smart-order#9

Exception in thread "main" java.lang.IllegalStateException

Ich mache:

cd /home/zeno/.software/aips2sqlite
git pull
cd /home/zeno/.software/aips2sqlite/jars
/usr/local/src/jre1.8.0_101/bin/java -jar -Xmx6000m /home/zeno/.software/aips2sqlite/jars/aips2sqlite.jar --owner=Axapharm --plain --xml --lang=de --pinfo

und erhalte

- Processing packages xlsx... Exception in thread "main" java.lang.IllegalStateException: Cannot get a text value from a numeric cell
        at org.apache.poi.xssf.usermodel.XSSFCell.typeMismatch(XSSFCell.java:845)
        at org.apache.poi.xssf.usermodel.XSSFCell.getRichStringCellValue(XSSFCell.java:294)
        at org.apache.poi.xssf.usermodel.XSSFCell.getStringCellValue(XSSFCell.java:246)
        at com.maxl.java.aips2sqlite.RealPatientInfo.extractPackageInfo(RealPatientInfo.java:159)
        at com.maxl.java.aips2sqlite.RealPatientInfo.process(RealPatientInfo.java:501)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.generateSQLiteDB(Aips2Sqlite.java:360)
        at com.maxl.java.aips2sqlite.Aips2Sqlite.main(Aips2Sqlite.java:334)

Fi Error: Nicht übereinstimmender Tag

#50 hat noch einen Folge Error:
Etwas mit den Tags stimmt noch nicht.
Wir haben auch noch sehr viele solche Dinge in der XML Datei: &#xa0
Diese Validierung schlägt fehl: xmllint --noout fi_de.xml

Error in parsing Kunden_alle.csv

The file Kunden_alle.csv is malformed, resulting in (at least) one record being wrong.

The problem is that one doctor contains a semicolon ; within the string that represents the doctor name (column M). This would be allowed if the whole name was inside double quotes, but that's not the case. The field separator for the file is the same ; consequently the parsing becomes corrupted.

You can verify the error for the doctor with GLN 7601000275400.

As a result the output file rose_conditions.ser contains bad data.

build aips2sqlite on Windows from commandline

Run gradlew.bat to install gradle.
Install make with https://chocolatey.org/docs/installation
Adapt Makefile, line 4 and remove ./ and put gradlew.bat instead of ./gradlew
Install JDK 1.8.0_211
add gradle.properties file and add the line org.gradle.java.home=C:\\Program Files\\Java\\jdk1.8.0_211
run make aips2sqlite

Fix IntelliJ Setup

Since being able to build aips2sqlite with make aips2sqlite (which is truly great and very useful and cool) the Intellij Setup no longer works. aips2sqlitecan no longer be built from Intellij Community Edition. @cybermax likes to work with Intellij, so lets try to fix it.

Skip products with pharmacode bigger then 7900000

If a Pharmacode in column A of file Artikelstamm_Vollstamm.csv is bigger then 7900000, skip the product (do not import it into rose_db_new_full.db or rose_db_new_atc_only.db).

zdavatz / aips2sqlite Goto Github PK

aips2sqlite's Issues

Recommend Projects

Recommend Topics

Recommend Org