sahandha/eif

Extended Isolation Forest for Anomaly Detection

eif's Issues

It's showing wrong outliers in values in the df['anomaly_scores'] < (df['Mean'] - (2*df['Std']) & df['anomaly_scores'] > (df['Mean'] + (2*df['Std')]

I am facing an issue in calculation of anomaly score which is less than or greater than (mean +- 2 Standard Deviation). It is not taking all the points , always, I think the reason is it is calculating all the path distances randomly and therefore missing few of them and causing problems. eg : You can see in outlier_2SD and outlier_3SD that in bottom of 2SD that is 'mean + 2SD ' is showing 4 No's which should have Yes.

| state_nm | state_code | district_nm | district_code | round | hh | women | men | rural_hh | v14 | v16 | v21 | v30 | v31 | v33 | Constant | outlier_2SD | outlier_3SD

121	Meghalaya	17	East Khasi Hills	298	NFHS5	918	1066	131	54.700001	93.699997	14.6	10.6	22.700001		57.5	1	Yes	Yes
94	Manipur	14	Ukhrul	279	NFHS5	865	711	106	85.900002	86.900002	11.1	12.3	6.6999998	41.400002	38.799999	1	Yes	Yes
118	Meghalaya	17	West Khasi Hills	296	NFHS5	924	1197	178	85.699997	87.400002	30.299999	15.3	43.400002	72.800003	64	1	Yes	Yes
119	Meghalaya	17	South West Khasi Hills	296	NFHS5	924	1203	196	100	85.900002	21.6	15.7	42.799999	82.300003	63.400002	1	Yes	Yes
92	Manipur	14	Imphal West	277	NFHS5	885	893	129	38.599998	92.5	15.3	16.4	7.0999999	35.400002	93.400002	1	Yes	No
93	Manipur	14	Imphal East	278	NFHS5	896	1021	161	59.799999	90	15.2	16.9	6.3000002	44.5	87.099998	1	Yes	No
87	Manipur	14	Sepati	272	NFHS5	859	837	120	95.099998	81.900002	15.5	17.5	6.4000001	44.900002	64.900002	1	No	No
122	Meghalaya	17	West Jaintia Hills	299	NFHS5	924	1172	155	90.5	80	19.9	17.5	31.4	70.400002	43.099998	1	No	No
115	Meghalaya	17	North Garo Hills	294	NFHS5	923	1283	180	88.099998	86.099998	13.2	17.700001	8	70.400002	24.299999	1	No	No
91	Manipur	14	Thoubal	276	NFHS5	893	1069	149	63.900002	85.400002	17.5	17.799999	2	54.200001	89.900002	1	No	No
120	Meghalaya	17	Ribhoi	297	NFHS5	927	1238	145	90.5	89.5	20.299999	19.700001	34.200001	71.099998	61	1	No	No
41	Bihar	10	Kishanganj	210	NFHS5	929	1116	111	90.599998	48	36.599998	21	12.2	43.299999	17.1	1	No	No
96	Mizoram	15	Aizawl	281	NFHS5	876	894	115	21.4	98.900002	3.2	21	12	44.400002	68.5	1	No	No
90	Manipur	14	Bishnupur	275	NFHS5	907	1043	140	64.199997	88.400002	20.9	21.6	8.6000004	42.400002	77.400002	1	No	No
102	Mizoram	15	Saiha	287	NFHS5	891	890	134	55.400002	95.099998	11.8	21.700001	22.200001	43.900002	35.5	1	No	No
88	Manipur	14	Tamenglong	273	NFHS5	882	839	133	85.199997	76.800003	19.9	22.200001	4	52.5	56.400002	1	No	No
89	Manipur	14	Churachandpur	274	NFHS5	847	872	119	92.900002	84.199997	10.5	22.6	5	48.099998	61.299999	1	No	No
95	Manipur	14	Chandel	280	NFHS5	847	757	105	87.800003	80.199997	22.299999	23.6	10.6	49.900002	66.599998	1	No	No
116	Meghalaya	17	East Jantia Hills	294	NFHS5	924	1311	200	100	76	25.299999	24	44.099998	81.099998	58.599998	1	No	No
42	Bihar	10	Purnia	211	NFHS5	904	973	133	88.699997	47.5	51.200001	24.299999	9.3999996	22	11.1	1	No	No
43	Bihar	10	Katihar	212	NFHS5	892	945	107	91.300003	49.700001	49.400002	26.4	22.6	57.5	15.3	1	No	No
10	J&K	1	Ganderbal	11	NFHS5	900	1202	158	83.400002	64	5.3000002	27.700001	7.1999998	36.099998	80.400002	1	No	No
7	J&K	1	Baramula	8	NFHS5	908	1329	166	81.199997	67.900002	3.8	28.200001	11.9	55.900002	75.199997	1	No	No
196	Gujarat	24	Kheda	483	NFHS5	890	1029	183	76.699997	70.900002	49.200001	28.700001	13.8	49.299999	61.200001	1	No	No
99	Mizoram	15	Lawngtlai	284	NFHS5	923	975	153	83.300003	76	16	29.1	11.3	59.099998	33	1	No	No
293	Lakshadweep	31	Lakshadweep	587	NFHS5	921	1234	135		96.5	1.3200001	30.139999	14.79	85.040001	88.309998	1	No	No
140	Assam	18	Majuli	312	NFHS5	921	1052	146	100	83.400002	25.5	31.299999	25.1	85.599998	72.199997	1	No	No
50	Bihar	10	Saran	219	NFHS5	945	1229	137	91.599998	65.599998	26.200001	31.4	18	36.700001	30.700001	1	No	No
112	Meghalaya	17	South West Garo Hills	293	NFHS5	921	1161	140	100	82.599998	15.3	32.299999	24.200001	82.900002	28.5	1	No	No
139	Assam	18	Jorhat	312	NFHS5	917	1019	148	76.599998	85.099998	24.9	32.299999	14.4	62.599998	67	1	No	No
150	Assam	18	Kamrup	321	NFHS5	894	1021	138	90.599998	79.599998	21.9	32.299999	17.1	58.900002	46.900002	1	No	No
84	Nagaland	13	Kiphire	269	NFHS5	922	799	125	78.599998	73.699997	22.5	32.700001	9.6999998	66.099998	5.8000002	1	No	No
100	Mizoram	15	Lunglei	285	NFHS5	915	973	128	57.599998	91.599998	4.8000002	33	13	62.799999	56.700001	1	No	No
117	Meghalaya	17	South Garo Hills	295	NFHS5	921	1088	143	90.5	88.599998	10.2	33.299999	20.299999	63.700001	33.299999	1	No	No
8	J&K	1	Bandipore	9	NFHS5	904	1263	177	83.300003	66.599998	1.8	34.599998	12	61.099998	82.599998	1	No	No
54	Bihar	10	Khagaria	223	NFHS5	909	1072	125	95.300003	51.799999	44.900002	34.900002	14.4	56.200001	17.4	1	No	No
141	Assam	18	Golaghat	313	NFHS5	919	1028	129	90.5	75.099998	20.700001	35.700001	25.299999	63.299999	65.699997	1	No	No
203	Gujarat	24	Bharuch	488	NFHS5	856	919	124	68.300003	74	16.799999	35.700001	16.200001	72.199997	65.699997	1	No	No
114	Meghalaya	17	East Garo Hills	294	NFHS5	919	1280	198	85.599998	86.699997	13.7	35.799999	22.1	79.199997	25.5	1	No	No
155	Assam	18	Udalguri	326	NFHS5	916	1088	151	95.400002	70	32	36.200001	14.6	47.200001	49.799999	1	No	No
135	Assam	18	Tinsukia	309	NFHS5	919	1062	155	81.099998	70.5	19.799999	36.5	17.6	72.199997	63.099998	1	No	No
154	Assam	18	Darrang	325	NFHS5	918	1053	176	92.800003	75.599998	42.799999	36.5	18	51.200001	37.5	1	No	No
82	Nagaland	13	Tuensang	267	NFHS5	924	987	142	81	77.800003	10.4	37.400002	7.9000001	51.700001	4.4000001	1	No	No
177	Gujarat	24	BasKantha	469	NFHS5	889	1049	157	85.900002	63.700001	37.299999	37.5	25.1	58.700001	56.099998	1	No	No
45	Bihar	10	Saharsa	214	NFHS5	907	960	110	92.300003	43.099998	51	37.599998	21.799999	38.700001	11.7	1	No	No
98	Mizoram	15	Kolasib	283	NFHS5	918	926	145	45.400002	96.900002	13.7	37.900002	18.700001	73.900002	66	1	No	No
145	Assam	18	Cachar	316	NFHS5	907	1110	152	81.199997	77.199997	29.9	38.200001	23.6	82.199997	32.700001	1	No	No
80	Nagaland	13	Dimapur	265	NFHS5	912	1053	155	48.299999	86.599998	4.4000001	38.400002	7.3000002	36.700001	50.099998	1	No	No
307	Kerala	32	Thiruvanthapuram	601	NFHS5	833	657	84	46.599998	98.5	6.1999998	38.5	18.6	33.900002	55.299999	1	No	No
152	Assam	18	lbari	323	NFHS5	886	1011	145	88	83.900002	28.1	38.700001	18.799999	60.599998	55.900002	1	No	No
103	Mizoram	15	Serchhip	288	NFHS5	915	917	134	50	99.699997	7.1999998	39.400002	17.299999	59.799999	60.299999	1	No	No
38	Bihar	10	Madhubani	207	NFHS5	918	976	83	95.5	53.200001	39.200001	39.5	18.6	54.299999	34.900002	1	No	No
48	Bihar	10	Gopalganj	217	NFHS5	912	1139	96	93.400002	63.299999	28	39.5	17.9	45.700001	27.9	1	No	No
138	Assam	18	Sivasagar	311	NFHS5	919	1020	146	88	86.599998	27.9	39.5	22.1	80.099998	80.900002	1	No	No
113	Meghalaya	17	West Garo Hills	293	NFHS5	923	1090	158	85.699997	89	10.4	39.900002	32	58.099998	48	1	No	No
34	Bihar	10	Pashchim Champaran	203	NFHS5	934	1070	109	91.400002	52.5	39.099998	40.200001	21	38.900002	25.4	1	No	No
110	Tripura	16	Ukoti	292	NFHS5	904	950	121	92.800003	76.400002	38	40.200001	5.4000001	21.9	24.6	1	No	No
108	Tripura	16	South Tripura	290	NFHS5	888	866	96	90.800003	80.300003	46.200001	40.299999	13	28.299999	58.400002	1	No	No
303	Kerala	32	Kottayam	597	NFHS5	867	659	100	71.900002	99.699997	1.6	40.299999	17.5	40.799999	55.099998	1	No	No
194	Gujarat	24	Bhavgar	481	NFHS5	871	950	145	57.299999	74.199997	18	40.5	12.6	56.200001	70.900002	1	No	No
35	Bihar	10	Purbi Champaran	204	NFHS5	934	1142	87	91.199997	50.200001	49.200001	40.799999	18.4	45.700001	21.700001	1	No	No
37	Bihar	10	Sitamarhi	206	NFHS5	911	1004	116	93.199997	51.700001	46.799999	41	17.299999	52.200001	20.299999	1	No	No
51	Bihar	10	Vaishali	220	NFHS5	925	1112	123	93.199997	62.400002	44.900002	41	18.5	47.400002	24.5	1	No	No
143	Assam	18	Karbi Alog	314	NFHS5	919	1042	156	85.599998	78.800003	26.1	41.099998	23.6	74.599998	63.900002	1	No	No
189	Gujarat	24	Devbhumi Dwarka	477	NFHS5	915	1170	198	69.099998	66.099998	11.6	41.099998	33.5	65.900002	76.599998	1	No	No
101	Mizoram	15	Mamit	286	NFHS5	917	898	146	83.300003	89.599998	16.799999	41.299999	15.8	62.599998	52.5	1	No	No
170	West Bengal	19	Puruliya	340	NFHS5	912	1050	138	87.900002	61	37	41.299999	21.200001	30.700001	57.799999	1	No	No
151	Assam	18	Kamrup metropolitan	322	NFHS5	903	926	165	17.1	86	21.9	41.5	16.6	50.299999	68.900002	1	No	No
228	Maharashtra	27	Parbhani	513	NFHS5	860	888	158	69.5	73.400002	48	41.5	17.4	48.299999	47.400002	1	No	No
298	Kerala	32	Malappuram	592	NFHS5	905	1004	141	54.400002	99.199997	15.3	41.5	10.9	61.299999	90.400002	1	No	No
3	Leh	1	Kargil	4	NFHS5	909	1201	166	88	77.199997	2.5	41.599998	9.1000004	55.400002	88	1	No	No
179	Gujarat	24	Mahesa	471	NFHS5	839	883	136	73.800003	75.099998	32.299999	41.799999	20.9	76.400002	56.599998	1	No	No
86	Nagaland	13	Peren	271	NFHS5	923	943	148	85.699997	77.599998	9.1000004	42	8.1999998	75.900002	14.5	1	No	No
61	Bihar	10	Pat	230	NFHS5	897	942	143	57.200001	67.599998	26.6	42.299999	18.6	46.400002	17.9	1	No	No
192	Gujarat	24	Jugadh	479	NFHS5	886	963	164	61.900002	82.800003	11.2	42.299999	26.5	69	72.5	1	No	No
49	Bihar	10	Siwan	218	NFHS5	941	1256	95	95.599998	70.900002	21.299999	42.400002	9.3000002	40.200001	30.299999	1	No	No
40	Bihar	10	Araria	209	NFHS5	947	1122	124	93.5	43.700001	52	42.799999	11	33.900002	25.799999	1	No	No
71	Bihar	10	Arwal	240	NFHS5	966	1185	133	93.300003	62.5	37.5	42.799999	27.1	56	34.5	1	No	No
75	Sikkim	11	East Sikkim	244	NFHS5	848	724	93	58.5	90.099998	9.3999996	42.799999	17	52.900002	43.799999	1	No	No
142	Assam	18	West Karbi Alog	314	NFHS5	921	1065	146	92.800003	73.900002	21.299999	43.299999	23	80.5	46.700001	1	No	No
133	Assam	18	Lakhimpur	307	NFHS5	916	957	140	90.5	83.900002	36.299999	43.599998	30.799999	80.300003	51.700001	1	No	No
136	Assam	18	Dibrugarh	310	NFHS5	920	1086	161	81.099998	76.599998	23	43.599998	28.9	76.5	75.599998	1	No	No
182	Gujarat	24	Gandhigar	473	NFHS5	862	919	151	58.5	81.800003	32.599998	43.599998	16.9	59.799999	71	1	No	No
214	Maharashtra	27	Jalgaon	499	NFHS5	842	881	155	69.5	76.5	28	43.599998	9.3999996	47	58.400002	1	No	No
137	Assam	18	Charaideo	311	NFHS5	922	1117	166	92.800003	71.900002	22.6	43.900002	22	69.5	64.5	1	No	No
106	Tripura	16	Sepahijala	289	NFHS5	888	892	121	90.800003	79.599998	51.900002	44.200001	9.8000002	39.599998	52.799999	1	No	No
63	Bihar	10	Buxar	232	NFHS5	975	1295	158	91.099998	68.099998	30.799999	44.599998	31.4	67.300003	27.4	1	No	No
183	Gujarat	24	Botad	474	NFHS5	866	1032	149	68.900002	71.800003	13	44.599998	17.4	64.099998	82.699997	1	No	No
270	Karnataka	29	Haveri	564	NFHS5	875	1060	141	78.900002	71.5	16.5	44.599998	20.799999	55.799999	58.700001	1	No	No
144	Assam	18	Dima Hasao	315	NFHS5	918	1001	149	71.699997	87.699997	16.5	44.900002	18.299999	66.599998	46.900002	1	No	No
134	Assam	18	Dhemaji	308	NFHS5	918	989	148	92.800003	81.5	32	45	28.200001	67	62.200001	1	No	No
68	Bihar	10	wada	237	NFHS5	963	1239	121	91	62.099998	43.299999	45.099998	13.5	45.700001	31.6	1	No	No
70	Bihar	10	Jehabad	239	NFHS5	949	1072	156	88.5	63.200001	41.599998	45.299999	40.700001	61.5	17.4	1	No	No
148	Assam	18	Bongaigaon	319	NFHS5	916	1092	169	85.599998	75.800003	41.700001	45.299999	23.4	65.099998	33.900002	1	No	No
28	Himachal Pradesh	2	Una	29	NFHS5	860	876	104	90.599998	92.5	1.6	45.900002	14.8	71.400002	71.699997	1	No	No
230	Maharashtra	27	Aurangabad	515	NFHS5	875	1011	147	57.099998	83.099998	35.799999	46	11.8	44.400002	57.200001	1	No	No
11	J&K	1	Pulwama	12	NFHS5	906	1155	151	85.900002	76.599998	0.5	46.099998	4.3000002	55.099998	96.199997	1	No	No
53	Bihar	10	Begusarai	222	NFHS5	966	1170	165	79.900002	62.299999	49.5	46.200001	17.4	53.099998	21.6	1	No	No
294	Kerala	32	Kasaragod	588	NFHS5	922	945	116	61.900002	95.900002	4.6999998	46.599998	23.9	69.099998	91.199997	1	No	No
85	Nagaland	13	Kohima	270	NFHS5	923	817	132	54.700001	95.199997	1	46.700001	11.6	51.5	28.299999	1	No	No
55	Bihar	10	Bhagalpur	224	NFHS5	966	1154	179	79.900002	65.599998	42.400002	46.799999	31.200001	39.900002	27.6	1	No	No
129	Assam	18	gaon	305	NFHS5	922	1122	169	88.199997	78.400002	42.599998	46.799999	16.6	73.699997	59.400002	1	No	No
273	Karnataka	29	Davagere	567	NFHS5	840	973	133	68.599998	76	19.1	46.900002	16.200001	45	63.099998	1	No	No
81	Nagaland	13	Phek	266	NFHS5	919	931	154	85.800003	85.800003	6.5	47.099998	11.3	63.099998	9.5	1	No	No
153	Assam	18	Baksa	324	NFHS5	921	1175	156	97.599998	74.800003	24.9	47.299999	24.799999	76.199997	56	1	No	No
127	Assam	18	Barpeta	303	NFHS5	914	1163	159	90.699997	74.800003	40.099998	47.400002	18.4	55.099998	43.599998	1	No	No
44	Bihar	10	Madhepura	213	NFHS5	931	1055	127	95.800003	47.599998	52	47.599998	29.9	60.799999	20.9	1	No	No
69	Bihar	10	Jamui	238	NFHS5	979	1172	119	91.099998	48.700001	51.900002	47.700001	18.4	64.400002	37.900002	1	No	No
17	J&K	1	Kishtwar	18	NFHS5	895	1158	152	93	71.300003	7.1999998	47.799999	3.0999999	45.599998	76.400002	1	No	No
62	Bihar	10	Bhojpur	231	NFHS5	974	1213	181	86.699997	64.099998	31.200001	47.900002	17.5	45	33.5	1	No	No
175	West Bengal	19	Purba Medinipur	345	NFHS5	873	957	131	88.300003	77	57.599998	48.099998	20.6	50	56.799999	1	No	No
130	Assam	18	Hojai	305	NFHS5	896	1027	148	74.300003	83.199997	30.9	48.200001	21.700001	73.5	51.299999	1	No	No
107	Tripura	16	Gomati	290	NFHS5	889	851	109	80.800003	77.199997	42.799999	48.700001	13.1	40.200001	40.599998	1	No	No
46	Bihar	10	Darbhanga	215	NFHS5	913	1053	100	91.599998	49.400002	45.099998	48.799999	21.5	50.200001	24.9	1	No	No
210	Dadar Nagar & Daman & Diu	25	Daman	495	NFHS5	851	733	152	18	85.900002	22.799999	48.900002	20.700001	61.400002	70.5	1	No	No
229	Maharashtra	27	Jal	514	NFHS5	863	937	145	81.800003	71.800003	35	48.900002	15.7	43.299999	58.400002	1	No	No
57	Bihar	10	Munger	226	NFHS5	972	1100	157	71.099998	69.5	34.700001	49	23.299999	65.300003	36.799999	1	No	No
200	Gujarat	24	Vadodara	486	NFHS5	855	894	167	37.200001	84.599998	22.799999	49.099998	17.5	44.5	65.900002	1	No	No
312	Telanga	99	Komaram Bheem Asifabad	532	NFHS5	897	934	136	83.800003	51.700001	25	49.099998	12.5	33	68.800003	1	No	No
195	Gujarat	24	And	482	NFHS5	870	914	163	69.900002	75.900002	28	49.200001	23.9	74.5	64.199997	1	No	No
166	West Bengal	19	dia	336	NFHS5	919	1034	146	71.5	76.199997	39.900002	49.299999	20.9	61.599998	71.199997	1	No	No
15	J&K	1	Doda	16	NFHS5	877	1000	141	92.900002	69.199997	11	49.400002	10.8	58.599998	74.300003	1	No	No
147	Assam	18	Hailakandi	318	NFHS5	905	1057	130	93	82.199997	32.900002	49.5	26.1	94.699997	43.099998	1	No	No
97	Mizoram	15	Champhai	282	NFHS5	902	806	150	62.099998	97.699997	11	49.599998	24.9	70	63.400002	1	No	No
265	Karnataka	29	Raichur	559	NFHS5	891	1177	166	74.199997	54.299999	21.9	49.599998	19.200001	54.099998	67.5	1	No	No
36	Bihar	10	Sheohar	205	NFHS5	944	886	104	95.699997	52.5	34.599998	49.700001	24.700001	58.700001	25.299999	1	No	No
105	Tripura	16	Khowai	289	NFHS5	908	872	137	88.300003	80.5	28.299999	49.900002	8.6999998	34.700001	41.5	1	No	No
79	Nagaland	13	Wokha	264	NFHS5	921	789	124	78.599998	91.599998	3.2	50.099998	10.8	58.099998	34.700001	1	No	No
59	Bihar	10	Sheikhpura	228	NFHS5	958	1160	132	82.300003	55	46.099998	50.299999	26.200001	58.599998	28.4	1	No	No
213	Maharashtra	27	Dhule	498	NFHS5	838	883	137	74.300003	68.800003	40.5	50.299999	19.1	48.400002	63.200001	1	No	No
21	J&K	1	Samba	22	NFHS5	901	1121	150	83.5	84.800003	6.3000002	50.400002	7.1999998	63.599998	96.199997	1	No	No
58	Bihar	10	Lakhisarai	227	NFHS5	969	1222	147	86.599998	57.900002	56.099998	50.400002	19.799999	53.400002	28	1	No	No
310	Andaman & Nicobar	35	South Andaman	640	NFHS5	868	844	134	41.599998	86.699997	17.1	50.5	31.200001	88.199997	85.900002	1	No	No
191	Gujarat	24	Gir Somth	479	NFHS5	900	1158	185	74.099998	73.099998	9.8999996	51.099998	24	48.599998	80.699997	1	No	No
27	Himachal Pradesh	2	Hamirpur	28	NFHS5	881	797	87	93	94.800003	3.5	51.400002	17.299999	32.700001	59.400002	1	No	No
19	J&K	1	Resai	20	NFHS5	915	1100	140	90.400002	68.199997	9.6000004	51.700001	8	56	75.900002	1	No	No
169	West Bengal	19	Bankura	339	NFHS5	889	997	133	90.199997	68.300003	45.700001	51.900002	12.4	43.599998	75.699997	1	No	No
193	Gujarat	24	Amreli	480	NFHS5	870	936	127	75.199997	77.199997	10.5	51.900002	30	75.400002	92.599998	1	No	No
299	Kerala	32	Palakkad	593	NFHS5	912	881	113	76.199997	94.400002	14.1	51.900002	14.6	56.599998	68.400002	1	No	No
23	Himachal Pradesh	2	Kangra	24	NFHS5	887	930	123	95.199997	94.400002	1.5	52	15.8	52.200001	56.299999	1	No	No
9	J&K	1	Srigar	10	NFHS5	889	1030	138	2	78.400002	1.7	52.200001	9.6999998	78.699997	85.099998	1	No	No
209	Dadar Nagar & Daman & Diu	25	Diu	494	NFHS5	911	978	101	55.400002	90.800003	2.3	52.200001	35.400002	82.599998	90.699997	1	No	No
60	Bihar	10	landa	229	NFHS5	962	1151	146	84	56.099998	42	52.400002	27.799999	54.099998	29.299999	1	No	No
67	Bihar	10	Gaya	236	NFHS5	963	1153	147	86.699997	59.400002	42.799999	52.5	29.5	51.900002	25.1	1	No	No
77	Nagaland	13	Mokokchung	262	NFHS5	921	892	118	71.400002	94	6	52.5	11.4	70.300003	18.200001	1	No	No
16	J&K	1	Ramban	17	NFHS5	911	1192	182	95.199997	62.200001	5.5	52.599998	15.6	62.299999	79.199997	1	No	No
198	Gujarat	24	Panch Mahals	484	NFHS5	908	1101	196	83.199997	71.699997	34.099998	52.599998	29.200001	68.099998	88.699997	1	No	No
178	Gujarat	24	Patan	470	NFHS5	904	1011	169	78.5	71.099998	35.400002	52.900002	37.5	77.300003	79.800003	1	No	No
285	Karnataka	29	Gulbarga	579	NFHS5	909	1147	167	66.800003	68.199997	29.799999	53	32.299999	70.099998	53.599998	1	No	No
12	J&K	1	Shupiyan	13	NFHS5	912	1135	148	92.800003	80.800003	2.0999999	53.099998	8.6999998	49.799999	86.800003	1	No	No
125	Assam	18	Dhubri	301	NFHS5	912	1017	128	88.199997	69.5	50.799999	53.200001	22.6	76.699997	37.599998	1	No	No
4	J&K	1	Punch	5	NFHS5	918	1182	168	92.900002	79.300003	5.4000001	53.299999	15.1	70	86.400002	1	No	No
109	Tripura	16	Dhalai	291	NFHS5	920	979	130	90.5	71.599998	38.900002	53.299999	3.9000001	50.799999	52	1	No	No
66	Bihar	10	Aurangabad	235	NFHS5	970	1269	161	91.400002	67.599998	27.299999	53.5	26.6	52.799999	29.299999	1	No	No
186	Gujarat	24	Rajkot	476	NFHS5	882	979	159	39	84.699997	12.1	53.5	48.099998	86.199997	93.5	1	No	No
52	Bihar	10	Samastipur	221	NFHS5	923	1061	118	95.599998	54.299999	49.799999	53.799999	20.6	59.400002	23.5	1	No	No
104	Tripura	16	West Tripura	289	NFHS5	903	865	151	36.200001	86.199997	37.099998	53.900002	9.3999996	42.099998	61.200001	1	No	No
165	West Bengal	19	Paschim Barddhaman	335	NFHS5	912	1190	184	18.9	73.5	31.799999	54	13.7	25.799999	70.400002	1	No	No
231	Maharashtra	27	shik	516	NFHS5	860	1023	155	59.200001	80	29.6	54	10.7	55.700001	66.400002	1	No	No
311	Telanga	99	Adilabad	532	NFHS5	913	965	156	76.099998	64.800003	21.4	54.099998	12	32.599998	74.800003	1	No	No
2	Leh	1	Leh (ladakh)	3	NFHS5	909	1154	141	67.099998	76.400002	2.5	54.200001	15.8	62	69.599998	1	No	No
295	Kerala	32	Kannur	589	NFHS5	919	950	101	35.599998	99.099998	5.4000001	54.200001	16.799999	77	78.300003	1	No	No
111	Tripura	16	North Tripura	292	NFHS5	909	1039	125	76.599998	83.099998	34.200001	54.400002	13.5	62.200001	67.099998	1	No	No
305	Kerala	32	Pathamthitta	599	NFHS5	854	625	92	88.800003	99.699997	0	54.400002	16.200001	63.099998	83.900002	1	No	No
161	West Bengal	19	Maldah	332	NFHS5	911	1113	150	85.599998	72.300003	49.099998	54.700001	18.9	46.5	82.800003	1	No	No
291	Goa	30	North Goa	585	NFHS5	925	975	148	40.799999	92.400002	7.5	54.799999	33.200001	82.099998	91.300003	1	No	No
146	Assam	18	Karimganj	317	NFHS5	917	1170	160	90.5	80.699997	27.700001	54.900002	31	84.5	42.799999	1	No	No
14	J&K	1	Kulgam	15	NFHS5	912	1190	139	81.099998	70.900002	3.7	55	5.6999998	63.299999	92.199997	1	No	No
126	Assam	18	Goalpara	302	NFHS5	917	1158	164	85.699997	74.099998	41.799999	55.099998	19.299999	68.099998	44.099998	1	No	No
313	Telanga	99	Mancherial	532	NFHS5	890	846	118	57.299999	69.5	14	55.299999	9.5	33	64.699997	1	No	No
181	Gujarat	24	SabarKantha	472	NFHS5	892	1018	152	83.900002	74.5	27	55.5	21.799999	64	73.199997	1	No	No
76	Nagaland	13	Mon	261	NFHS5	899	832	123	85.400002	78.199997	3	55.599998	11.2	79.800003	9.6999998	1	No	No
149	Assam	18	Chirang	320	NFHS5	912	1076	148	93.099998	71.5	30.9	55.599998	34.200001	75.300003	56.5	1	No	No
174	West Bengal	19	Paschim Medinipur	344	NFHS5	910	1002	137	88.099998	70.900002	55.700001	55.599998	23.5	49.299999	63.099998	1	No	No
199	Gujarat	24	Dahod	485	NFHS5	899	1221	158	90.400002	56.099998	29.9	55.599998	31.6	79.300003	70.900002	1	No	No
47	Bihar	10	Muzaffarpur	216	NFHS5	914	1095	94	91.5	63	32.900002	55.700001	20.799999	57.900002	28.200001	1	No	No
56	Bihar	10	Banka	225	NFHS5	979	1098	147	95.5	55.700001	49.400002	55.700001	21.700001	50	31.700001	1	No	No
65	Bihar	10	Rohtas	234	NFHS5	975	1302	148	84.400002	76.5	30.299999	55.700001	38.599998	59.099998	34.200001	1	No	No
321	Telanga	99	Medak	535	NFHS5	881	883	128	92.900002	57.700001	31.799999	55.700001	9.1000004	51.400002	68.5	1	No	No
237	Maharashtra	27	Pune	521	NFHS5	786	832	140	41.599998	89	24	55.900002	23.5	35.700001	68.599998	1	No	No
132	Assam	18	Biswath	306	NFHS5	901	1000	135	95.300003	69.199997	25.299999	56.099998	29.9	73.5	46.5	1	No	No
5	J&K	1	Rajouri	6	NFHS5	917	1183	181	92.800003	79.400002	12.2	56.200001	6.4000001	53.700001	71.900002	1	No	No
39	Bihar	10	Supaul	208	NFHS5	945	1153	112	95.699997	42.099998	55.900002	56.200001	26.1	44.5	30.9	1	No	No
83	Nagaland	13	Longleng	268	NFHS5	924	873	132	85.699997	82	11	56.299999	9.3999996	55	15.4	1	No	No
180	Gujarat	24	Aravali	472	NFHS5	901	1078	187	88	71.900002	27	56.299999	30	75.599998	73.800003	1	No	No
317	Telanga	99	Karimgar	534	NFHS5	874	853	114	69.300003	70.300003	11.9	56.400002	14.5	24.299999	69.599998	1	No	No
0	J&K	1	Kupwara	1	NFHS5	904	1233	175	88.099998	76	3.0999999	56.5	8.8999996	64.599998	89.199997	1	No	No
78	Nagaland	13	Zunheboto	263	NFHS5	924	778	103	81	87.099998	4	56.5	14.8	70.400002	11.2	1	No	No
319	Telanga	99	Jagitial	534	NFHS5	898	885	99	78.599998	62.400002	28.4	56.5	17.299999	24	81.900002	1	No	No
128	Assam	18	Morigaon	304	NFHS5	915	1071	136	93	78.699997	39.099998	56.700001	27	66.400002	42	1	No	No
239	Maharashtra	27	Bid	523	NFHS5	877	864	146	81.5	76.300003	43.700001	56.700001	14.5	48.5	56.799999	1	No	No
245	Maharashtra	27	Sindhudurg	529	NFHS5	861	722	115	89.300003	92.099998	5	56.799999	26.799999	63.799999	73.400002	1	No	No
315	Telanga	99	Nizamabad	533	NFHS5	891	891	107	72.099998	63.099998	23.700001	56.799999	8.6000004	60.400002	78.5	1	No	No
316	Telanga	99	Kamareddy	533	NFHS5	899	857	134	88.199997	58.599998	30.799999	56.799999	9.3999996	44.400002	79.5	1	No	No
304	Kerala	32	Alappuzha	598	NFHS5	886	685	111	45.200001	99.699997	3.8	56.900002	11	40.900002	65.699997	1	No	No
20	J&K	1	Jammu	21	NFHS5	889	1050	125	51.099998	91.5	5.3000002	57.099998	6.8000002	73.099998	95.400002	1	No	No
233	Maharashtra	27	Thane	517	NFHS5	755	756	91	16	90.5	18.4	57.099998	24.700001	56.900002	70.199997	1	No	No
308	Andaman & Nicobar	35	Nicobars	638	NFHS5	882	764	125	100	87.5	11.4	57.200001	40.400002	49.400002	71.699997	1	No	No
13	J&K	1	Antgar	14	NFHS5	906	1102	125	74.599998	74.699997	2.5	57.299999	18.799999	72	80.900002	1	No	No
335	Telanga	99	Warangal Rural	540	NFHS5	904	830	123	92.699997	60.099998	22.9	57.5	16.299999	53.5	70.400002	1	No	No
1	J&K	1	Badgam	2	NFHS5	911	1200	172	88.599998	74.099998	1.5	57.700001	13.7	55.400002	66.5	1	No	No
267	Karnataka	29	Gadag	561	NFHS5	892	1136	193	64.800003	70.699997	27.700001	57.700001	25.4	70	68.699997	1	No	No
123	Assam	18	Kokrajhar	300	NFHS5	897	1052	144	94.699997	73.699997	36.200001	58	18.700001	51.299999	36.900002	1	No	No
184	Gujarat	24	Ahmedabad	474	NFHS5	906	1010	155	14.6	81.5	17.5	58.200001	36.200001	76.5	77.800003	1	No	No
164	West Bengal	19	Purba Barddhaman	335	NFHS5	915	1088	163	85.800003	73.199997	50.400002	58.5	16	36	80.5	1	No	No
297	Kerala	32	Kozhikode	591	NFHS5	888	825	107	33.900002	99.099998	4.1999998	58.5	14.4	71.5	91.800003	1	No	No
314	Telanga	99	Nirmal	532	NFHS5	899	851	107	79.099998	58.599998	23.299999	58.5	9.1999998	39.400002	69.699997	1	No	No
187	Gujarat	24	Morbi	476	NFHS5	902	1119	182	62.200001	82.400002	8.8999996	58.700001	44.400002	82	75.099998	1	No	No
234	Maharashtra	27	Mumbai Suburban	518	NFHS5	635	552	64	0	91.599998	10	58.700001	20.4	48.200001	72.199997	1	No	No
131	Assam	18	Sonitpur	306	NFHS5	909	1067	148	88.5	76.900002	24	58.900002	20.5	72	45.5	1	No	No
160	West Bengal	19	Dakshin Dijpur	331	NFHS5	920	1137	163	85.800003	74.300003	45.599998	59	16	68.300003	76.400002	1	No	No
197	Gujarat	24	Mahisagar	484	NFHS5	899	1017	168	90.900002	70.900002	30.700001	59.099998	44.299999	83.099998	76.599998	1	No	No
318	Telanga	99	Rajan Sircilla	534	NFHS5	907	901	125	78.800003	64.699997	13.2	59.299999	18.9	60	67.099998	1	No	No
323	Telanga	99	Siddipet	535	NFHS5	883	832	116	86.099998	71	19	59.400002	11.6	46.700001	62.599998	1	No	No
64	Bihar	10	Kaimur (Bhabua)	233	NFHS5	973	1167	146	95.599998	66	27.1	59.700001	45.099998	62.599998	25.6	1	No	No
281	Karnataka	29	Dakshi Kanda	575	NFHS5	867	987	149	53.5	92.699997	4.9000001	60.200001	34.799999	67.900002	82	1	No	No
244	Maharashtra	27	Ratgiri	528	NFHS5	869	807	127	83	87.199997	4.4000001	60.5	22	57.5	78.599998	1	No	No
306	Kerala	32	Kollam	600	NFHS5	859	753	78	55.900002	98.199997	1.8	60.5	15.1	62	79.300003	1	No	No
124	Assam	18	South Salmara Mancachar	301	NFHS5	912	1085	162	95.199997	63.5	44.700001	60.799999	26.700001	71.800003	35.299999	1	No	No
159	West Bengal	19	Uttar Dijpur	330	NFHS5	922	1129	157	88.099998	65.400002	30.299999	60.900002	10.3	50.599998	70.099998	1	No	No
72	Sikkim	11	North Sikkim	241	NFHS5	882	768	112	90.800003	81.400002	16	61	28.6	50.400002	59.400002	1	No	No
167	West Bengal	19	North Twenty Four Parga	337	NFHS5	924	1055	148	42.900002	85.5	33.599998	61	21.9	66.5	89.900002	1	No	No
212	Maharashtra	27	ndurbar	497	NFHS5	897	1040	171	82.900002	57.700001	24	61.200001	11.8	50.299999	58.200001	1	No	No
168	West Bengal	19	Hugli	338	NFHS5	892	1020	136	62.799999	77.400002	40.799999	61.299999	13.3	59.700001	72.199997	1	No	No
300	Kerala	32	Thrissur	594	NFHS5	852	715	66	33.099998	99.400002	1	61.5	10.1	59.700001	82.400002	1	No	No
301	Kerala	32	Erkulam	595	NFHS5	857	747	115	31.700001	99.300003	2.9000001	61.5	12.9	79.900002	82.199997	1	No	No
322	Telanga	99	Sangareddy	535	NFHS5	892	911	119	63.799999	63.599998	30.6	61.700001	10.2	47.5	66.400002	1	No	No
206	Gujarat	24	Valsad	491	NFHS5	882	1031	159	63.599998	82.900002	19.4	61.900002	52.700001	70.5	92.5	1	No	No
271	Karnataka	29	Bellary	565	NFHS5	883	1103	156	61.400002	64.400002	22.200001	62.299999	33.5	54.799999	56.400002	1	No	No
176	Gujarat	24	Kachchh	468	NFHS5	922	1026	181	64.300003	75.599998	19	62.5	36.099998	85.800003	84.199997	1	No	No
263	Karnataka	29	Bijapur	557	NFHS5	887	1091	149	76.400002	66.599998	39.200001	62.700001	23.799999	62.099998	56.400002	1	No	No
266	Karnataka	29	Koppal	560	NFHS5	872	1017	171	83.800003	59.799999	27.1	63.200001	15.6	59	50.700001	1	No	No
286	Karnataka	29	Yadgir	580	NFHS5	904	1242	171	81.099998	48.099998	33.200001	63.700001	23.1	65	63.599998	1	No	No
18	J&K	1	Udhampur	19	NFHS5	908	1148	150	80.800003	77.300003	6.0999999	63.900002	9	57.799999	69.800003	1	No	No
211	Dadar Nagar & Daman & Diu	26	Dadar & Nagar	496	NFHS5	914	1002	174	52.099998	72.800003	30	64.300003	26.4	71.199997	90.900002	1	No	No
338	Telanga	99	Jayashankar Bhupalapally	540	NFHS5	897	852	124	93.099998	58.900002	24.9	64.300003	13.5	47.099998	72.699997	1	No	No
336	Telanga	99	Warangal Urban	540	NFHS5	888	900	138	31.299999	76.900002	22.700001	64.400002	27.799999	49.400002	71.300003	1	No	No
247	Maharashtra	27	Sangli	531	NFHS5	898	966	152	74.400002	90.300003	27	64.699997	27.6	56.200001	80.099998	1	No	No
320	Telanga	99	Peddapalli	534	NFHS5	892	861	142	62.400002	73.699997	13.6	64.800003	16.9	44.5	77.900002	1	No	No
162	West Bengal	19	Murshidabad	333	NFHS5	917	1144	159	81.099998	67.599998	55.400002	64.900002	6.4000001	60.900002	66.900002	1	No	No
205	Gujarat	24	vsari	490	NFHS5	893	991	162	69.300003	84.099998	15.7	65.099998	62.799999	87.5	94.699997	1	No	No
22	Himachal Pradesh	2	Chamba	23	NFHS5	907	931	109	93.099998	84.099998	3.8	65.199997	31.799999	71.699997	62.700001	1	No	No
262	Karnataka	29	Bagalkot	556	NFHS5	881	1138	182	68.900002	69.699997	38.700001	65.300003	15.8	51.099998	76.199997	1	No	No
30	Himachal Pradesh	2	Solan	31	NFHS5	883	855	148	83.800003	91	13.3	65.5	21.799999	71.599998	88.199997	1	No	No
207	Gujarat	24	Surat	492	NFHS5	902	986	195	21.799999	84.199997	13.1	65.599998	39.799999	83.599998	93.400002	1	No	No
185	Gujarat	24	Surendragar	475	NFHS5	869	931	143	72.5	75.099998	19.5	65.699997	25.1	75.5	57.5	1	No	No
268	Karnataka	29	Dharwad	562	NFHS5	864	1051	156	45.700001	81.800003	17.799999	65.699997	29.700001	79.5	85.199997	1	No	No
232	Maharashtra	27	Palghar	517	NFHS5	890	941	131	49	77.599998	14.6	66	21.6	53	86.300003	1	No	No
24	Himachal Pradesh	2	Lahul & spiti	25	NFHS5	875	698	113	100	86.199997	11.2	66.300003	16.4	46.400002	65.599998	1	No	No
251	Andhra Pradesh	28	East Godavari	545	NFHS5	888	824	105	74.800003	77.900002	26	66.300003	12.5	32.400002	51	1	No	No
188	Gujarat	24	Jamgar	477	NFHS5	920	1042	165	47.5	83.099998	6.8000002	66.5	60.799999	92.199997	73.800003	1	No	No
156	West Bengal	19	Darjiling	327	NFHS5	898	1058	163	60.099998	77	21.200001	67	11.8	49.200001	79.199997	1	No	No
202	Gujarat	24	rmada	487	NFHS5	904	1021	155	90.400002	67.300003	29.5	67	45	75.599998	83.400002	1	No	No
201	Gujarat	24	Chhota Udaipur	486	NFHS5	896	1035	173	93.099998	53.200001	27.5	67.300003	64.199997	91.199997	81.199997	1	No	No
226	Maharashtra	27	nded	511	NFHS5	908	1026	164	74.800003	71.900002	32.200001	67.300003	15.3	49.900002	53.5	1	No	No
238	Maharashtra	27	Ahmadgar	522	NFHS5	883	993	139	81.5	86.199997	26.9	67.400002	19.5	31.700001	76.599998	1	No	No
25	Himachal Pradesh	2	Kullu	26	NFHS5	883	892	128	90.800003	87.400002	8.6999998	67.5	16.799999	69.699997	60	1	No	No
74	Sikkim	11	South Sikkim	243	NFHS5	880	745	114	86	90.5	9.8999996	67.599998	24.299999	67.199997	84.5	1	No	No
259	Andhra Pradesh	28	Antapur	553	NFHS5	882	868	146	71.400002	63.599998	37.299999	67.599998	21.9	36.400002	66.599998	1	No	No
158	West Bengal	19	Koch Bihar	329	NFHS5	920	1095	157	90.400002	79.199997	46.700001	67.699997	18.299999	49.700001	77.300003	1	No	No
250	Andhra Pradesh	28	Visakhapatm	544	NFHS5	869	818	112	53.599998	69.5	25.4	67.699997	15.2	35.299999	58.599998	1	No	No
292	Goa	30	South Goa	586	NFHS5	931	1055	165	36.099998	93.800003	3.9000001	67.800003	19.700001	89.400002	96	1	No	No
302	Kerala	32	Idukki	596	NFHS5	857	710	121	95.199997	94.400002	7.0999999	67.800003	22.299999	64.300003	74.300003	1	No	No
173	West Bengal	19	South Twenty Four Parga	343	NFHS5	917	1089	148	74.099998	85.599998	41.900002	67.900002	17.700001	54.5	87.5	1	No	No
217	Maharashtra	27	Washim	502	NFHS5	903	986	196	83.400002	78	27.700001	68.300003	14.2	37.900002	60	1	No	No
171	West Bengal	19	Haora	341	NFHS5	916	1067	153	36.099998	80.5	30.4	68.400002	20.5	56.599998	82.699997	1	No	No
257	Andhra Pradesh	28	Y.S.R.	551	NFHS5	910	1017	140	66.900002	63.799999	25.6	68.400002	29.5	14.6	82.800003	1	No	No
190	Gujarat	24	Porbandar	478	NFHS5	917	1018	169	52.799999	84.300003	10	68.5	59.099998	81	92.099998	1	No	No
264	Karnataka	29	Bidar	558	NFHS5	914	1181	172	74.400002	73.800003	19.200001	68.5	40.700001	81.900002	55.299999	1	No	No
255	Andhra Pradesh	28	Prakasam	549	NFHS5	821	689	95	81.699997	62.799999	37.299999	68.699997	25.4	18.9	73.400002	1	No	No
260	Andhra Pradesh	28	Chittoor	554	NFHS5	865	828	121	72.300003	69.300003	28.1	68.900002	21.299999	29.1	65.300003	1	No	No
204	Gujarat	24	The Dangs	489	NFHS5	897	901	130	88.300003	68.900002	30.200001	69.099998	39.299999	76.5	90.199997	1	No	No
246	Maharashtra	27	Kolhapur	530	NFHS5	887	986	150	69.699997	90.699997	21	69.199997	27.9	52.400002	81.800003	1	No	No
269	Karnataka	29	Uttara Kanda	563	NFHS5	897	1000	164	71.699997	84.300003	11.6	69.300003	42.5	63.799999	57.900002	1	No	No
296	Kerala	32	Wayad	590	NFHS5	919	813	128	95.5	93.699997	8.3999996	69.5	11.8	68.5	94.199997	1	No	No
326	Telanga	99	Ranga Reddy	537	NFHS5	852	935	104	45.099998	72.099998	29	69.699997	18.700001	53.400002	75.099998	1	No	No
325	Telanga	99	Vikarabad	537	NFHS5	880	929	132	86	59.299999	39.799999	69.800003	8.6000004	36.5	54.799999	1	No	No
258	Andhra Pradesh	28	Kurnool	552	NFHS5	870	908	139	72.400002	57	36.900002	70	20.700001	22.200001	74.300003	1	No	No
324	Telanga	99	Hyderabad	536	NFHS5	703	642	84	0	83.599998	10.6	70	18.5	58.200001	69.900002	1	No	No
157	West Bengal	19	Jalpaiguri	328	NFHS5	919	1101	149	73.900002	73.599998	18.700001	70.099998	15.1	61.099998	88.400002	1	No	No
328	Telanga	99	Mahabubgar	538	NFHS5	907	1006	137	78.900002	59.599998	23.4	70.099998	31.4	48.299999	62.599998	1	No	No
29	Himachal Pradesh	2	Bilaspur	30	NFHS5	898	896	117	92.900002	91.199997	10	70.300003	25	49.799999	81.400002	1	No	No
337	Telanga	99	Jangoan	540	NFHS5	888	843	108	88.199997	65.5	20.299999	70.400002	19.5	33.900002	65.400002	1	No	No
261	Karnataka	29	Belgaum	555	NFHS5	907	1147	179	74.099998	74	32.799999	70.599998	29.4	66.400002	63.700001	1	No	No
73	Sikkim	11	West Sikkim	242	NFHS5	906	1034	150	95.099998	85.699997	14.1	70.800003	19.799999	69.099998	70.400002	1	No	No
236	Maharashtra	27	Raigarh	520	NFHS5	905	918	172	64.599998	79.199997	16	70.900002	22.4	64.699997	83.099998	1	No	No
249	Andhra Pradesh	28	Viziagaram	543	NFHS5	902	853	134	78.400002	58.299999	33.700001	71.199997	21.1	36.400002	71.400002	1	No	No
26	Himachal Pradesh	2	Mandi	27	NFHS5	902	878	112	93.199997	94	6.3000002	71.300003	14.2	44.799999	70.300003	1	No	No
275	Karnataka	29	Udupi	569	NFHS5	894	1065	145	72.5	90.300003	4.4000001	71.300003	42.299999	84.5	59.400002	1	No	No
172	West Bengal	19	Kolkata	342	NFHS5	879	921	138	0	87.599998	16.700001	71.699997	17.9	57	71.699997	1	No	No
235	Maharashtra	27	Mumbai	519	NFHS5	826	779	127	0	94.300003	4.5	71.699997	19.200001	77.800003	87.099998	1	No	No
248	Andhra Pradesh	28	Sri Potti Sriramulu Nellore	542	NFHS5	865	922	132	71.5	70.5	23.799999	71.699997	19.700001	21.700001	73	1	No	No
287	Karnataka	29	Kolar	581	NFHS5	883	990	164	70	78.300003	26.700001	71.900002	44.5	71.699997	90.900002	1	No	No
256	Andhra Pradesh	28	Srikakulam	550	NFHS5	874	780	100	83.199997	64.300003	25.4	72.199997	16	45.799999	78.400002	1	No	No
282	Karnataka	29	Kodagu	576	NFHS5	898	885	139	86	88.5	12.8	72.300003	42	70.800003	74.400002	1	No	No
327	Telanga	99	MedchalMalkajgiri	537	NFHS5	778	825	107	10.3	79.5	10.2	72.400002	25.4	57.900002	69.599998	1	No	No
272	Karnataka	29	Chitradurga	566	NFHS5	909	953	134	80.599998	75.599998	20.700001	72.5	63.400002	87	79.300003	1	No	No
6	J&K	1	Kathua	7	NFHS5	903	1064	149	85.900002	87.699997	1.4	72.699997	28.700001	74.5	31.6	1	No	No
216	Maharashtra	27	Akola	501	NFHS5	926	1098	202	60	87.5	13.5	72.800003	33.799999	59.5	76.300003	1	No	No
243	Maharashtra	27	Satara	527	NFHS5	896	978	149	80.699997	87.199997	18.1	72.800003	26.799999	68.099998	81.699997	1	No	No
227	Maharashtra	27	Hingoli	512	NFHS5	907	1121	173	86.099998	76.5	37.099998	72.900002	11.3	41.5	66.599998	1	No	No
289	Karnataka	29	Bangalore	583	NFHS5	837	840	125	9.8999996	87.300003	14.5	73	54.700001	90.699997	74.599998	1	No	No
309	Andaman & Nicobar	35	North & middle Andaman	639	NFHS5	874	789	108	97.699997	84	15.4	73.099998	23.200001	83.199997	79.199997	1	No	No
254	Andhra Pradesh	28	Guntur	548	NFHS5	851	807	93	67.199997	68.5	35.400002	73.199997	16.6	28.200001	62.5	1	No	No
163	West Bengal	19	Birbhum	334	NFHS5	922	1161	168	88.099998	70.800003	49.900002	73.900002	21.6	48.099998	78.400002	1	No	No
340	Telanga	99	Bhadradri Kothagudem	541	NFHS5	893	945	136	69.400002	68.699997	20.799999	74	18.299999	51.299999	70.099998	1	No	No
242	Maharashtra	27	Solapur	526	NFHS5	912	1003	153	67.199997	76.400002	40.299999	74.199997	35.099998	54.400002	81.900002	1	No	No
208	Gujarat	24	Tapi	493	NFHS5	904	991	144	90.800003	72	25.299999	74.400002	41.299999	71.5	91	1	No	No
31	Himachal Pradesh	2	Sirmaur	32	NFHS5	906	1044	167	88.300003	84.900002	5	74.5	24.9	76.5	85.199997	1	No	No
339	Telanga	99	Mahabubabad	540	NFHS5	888	903	121	90.5	58	28.299999	74.699997	16.299999	59.5	67.699997	1	No	No
331	Telanga	99	Jogulamba Gadwal	538	NFHS5	890	975	125	91	45	34.599998	74.900002	24.700001	43.299999	72.900002	1	No	No
333	Telanga	99	Yadadri Bhuvagiri	539	NFHS5	896	947	155	83.599998	68.400002	21.6	75	20.6	58.5	67.199997	1	No	No
221	Maharashtra	27	Bhandara	506	NFHS5	918	920	152	81.199997	89.099998	1.5	75.599998	30.5	51.599998	79	1	No	No
225	Maharashtra	27	Yavatmal	510	NFHS5	913	1001	177	78.800003	80.800003	11.7	75.699997	22.4	62.200001	66.900002	1	No	No
329	Telanga	99	Waparthy	538	NFHS5	902	965	146	83.599998	52.099998	32.599998	75.699997	22.299999	36.599998	62.799999	1	No	No
277	Karnataka	29	Tumkur	571	NFHS5	906	912	129	78.900002	81.900002	24.799999	75.800003	59.400002	76.099998	80.400002	1	No	No
32	Himachal Pradesh	2	Shimla	33	NFHS5	904	864	126	76.800003	93.400002	6.0999999	76	29.1	76.900002	85.300003	1	No	No
223	Maharashtra	27	Gadchiroli	508	NFHS5	921	915	164	88.400002	79.400002	10.1	76	32.700001	58.299999	86.800003	1	No	No
341	Telanga	99	Khammam	541	NFHS5	893	906	132	79.300003	66.300003	35	76.199997	25.799999	54.900002	76.300003	1	No	No
278	Karnataka	29	Bangalore Rural	572	NFHS5	891	956	142	73.599998	83.800003	14.1	76.300003	46.599998	85.800003	90.900002	1	No	No
274	Karnataka	29	Shimoga	568	NFHS5	902	1033	161	64.199997	79.800003	11.1	76.400002	42	74.699997	79.400002	1	No	No
330	Telanga	99	garkurnool	538	NFHS5	896	915	136	90.599998	57.099998	32.099998	76.5	27.1	58.299999	75.099998	1	No	No
33	Himachal Pradesh	2	Kinur	34	NFHS5	912	707	143	100	89.099998	27.9	76.699997	19.299999	67.699997	75.300003	1	No	No
241	Maharashtra	27	Osmabad	525	NFHS5	919	916	147	83.5	83.699997	36.599998	77.099998	28.5	58.099998	89.199997	1	No	No
240	Maharashtra	27	Latur	524	NFHS5	906	1006	160	74.099998	83.300003	31	77.199997	18.6	72.5	72.599998	1	No	No
252	Andhra Pradesh	28	West Godavari	546	NFHS5	884	841	122	80.5	77	22.1	77.199997	12.5	28.799999	62.700001	1	No	No
332	Telanga	99	lgonda	539	NFHS5	887	879	129	76.300003	62.599998	28.200001	77.199997	23.6	62.799999	65.699997	1	No	No
283	Karnataka	29	Mysore	577	NFHS5	857	951	134	61.099998	78.900002	17.5	77.300003	37.599998	78.599998	85.699997	1	No	No
288	Karnataka	29	Chikkaballapura	582	NFHS5	883	881	144	78.400002	76.300003	27.1	77.400002	49.700001	79.5	90.5	1	No	No
280	Karnataka	29	Hassan	574	NFHS5	905	979	136	79.5	82	16.200001	77.5	57.099998	82.199997	75.800003	1	No	No
290	Karnataka	29	Ramagara	584	NFHS5	878	814	108	76.800003	82.699997	11.8	77.5	34.099998	82.900002	88.699997	1	No	No
218	Maharashtra	27	Amravati	503	NFHS5	913	1060	180	65	87.800003	9.8000002	77.699997	20	52.799999	71.699997	1	No	No
222	Maharashtra	27	Gondiya	507	NFHS5	915	938	167	83.800003	87.5	6.5	77.800003	36.400002	61.700001	66.199997	1	Yes	No
334	Telanga	99	Suryapet	539	NFHS5	893	851	125	84.199997	63.5	29.5	77.900002	16.6	36.900002	70.400002	1	Yes	No
215	Maharashtra	27	Bulda	500	NFHS5	931	1055	189	78.599998	79.900002	24.1	78.099998	33.5	67.900002	72.699997	1	No	No
219	Maharashtra	27	Wardha	504	NFHS5	909	925	158	67	93	9	78.099998	36.5	52.799999	70.400002	1	No	No
224	Maharashtra	27	Chandrapur	509	NFHS5	922	965	175	64.599998	87.800003	9	78.099998	39.099998	57.400002	68.5	1	No	No
253	Andhra Pradesh	28	Krish	547	NFHS5	865	820	119	60.5	76.900002	25.299999	78.099998	16.1	27.700001	73.300003	1	No	No
279	Karnataka	29	Mandya	573	NFHS5	873	844	127	84.300003	78.300003	13.1	78.5	44.599998	77.300003	90.099998	1	Yes	Yes
284	Karnataka	29	Chamarajagar	578	NFHS5	890	957	136	82.900002	72.400002	19.299999	79.400002	54.799999	83.5	84.099998	1	Yes	Yes
276	Karnataka	29	Chikmagalur	570	NFHS5	885	945	143	79.400002	82.900002	19.5	79.599998	42.799999	74.400002	74.300003	1	Yes	Yes
220	Maharashtra	27	gpur	505	NFHS5	917	1063	169	31.200001	94.599998	7.0999999	81.199997	39.5	62.400002	71.400002	1	Yes	Yes

Installation problem

Hello, i'm trying to install this package, and i'm having error messages and i don't get to install it. Can you help?

Windows 10

(base) C:\Users\quirosgu>pip install eif
Collecting eif
Using cached eif-2.0.2.tar.gz (1.6 MB)
Requirement already satisfied: numpy in c:\users\quirosgu\anaconda3\lib\site-packages (from eif) (1.18.5)
Requirement already satisfied: cython in c:\users\quirosgu\anaconda3\lib\site-packages (from eif) (0.29.21)
Building wheels for collected packages: eif
Building wheel for eif (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\quirosgu\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\quirosgu\AppData\Local\Temp\pip-install-wz5r6gph\eif\setup.py'"'"'; file='"'"'C:\Users\quirosgu\AppData\Local\Temp\pip-install-wz5r6gph\eif\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\quirosgu\AppData\Local\Temp\pip-wheel-6t9epked'
cwd: C:\Users\quirosgu\AppData\Local\Temp\pip-install-wz5r6gph\eif
Complete output (19 lines):
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win32-3.8
copying eif_old.py -> build\lib.win32-3.8
copying version.py -> build\lib.win32-3.8
running egg_info
writing eif.egg-info\PKG-INFO
writing dependency_links to eif.egg-info\dependency_links.txt
writing requirements to eif.egg-info\requires.txt
writing top-level names to eif.egg-info\top_level.txt
reading manifest file 'eif.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'eif.egg-info\SOURCES.txt'
running build_ext
cythoning _eif.pyx to _eif.cpp
building 'eif' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/

ERROR: Failed building wheel for eif
Running setup.py clean for eif
Failed to build eif
Installing collected packages: eif
Running setup.py install for eif ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\quirosgu\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\quirosgu\AppData\Local\Temp\pip-install-wz5r6gph\eif\setup.py'"'"'; file='"'"'C:\Users\quirosgu\AppData\Local\Temp\pip-install-wz5r6gph\eif\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\quirosgu\AppData\Local\Temp\pip-record-fjpa9g_k\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\quirosgu\Anaconda3\Include\eif'
cwd: C:\Users\quirosgu\AppData\Local\Temp\pip-install-wz5r6gph\eif
Complete output (19 lines):
running install
running build
running build_py
creating build
creating build\lib.win32-3.8
copying eif_old.py -> build\lib.win32-3.8
copying version.py -> build\lib.win32-3.8
running egg_info
writing eif.egg-info\PKG-INFO
writing dependency_links to eif.egg-info\dependency_links.txt
writing requirements to eif.egg-info\requires.txt
writing top-level names to eif.egg-info\top_level.txt
reading manifest file 'eif.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'eif.egg-info\SOURCES.txt'
running build_ext
skipping '_eif.cpp' Cython extension (up-to-date)
building 'eif' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Users\quirosgu\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\quirosgu\AppData\Local\Temp\pip-install-wz5r6gph\eif\setup.py'"'"'; file='"'"'C:\Users\quirosgu\AppData\Local\Temp\pip-install-wz5r6gph\eif\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\quirosgu\AppData\Local\Temp\pip-record-fjpa9g_k\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\quirosgu\Anaconda3\Include\eif' Check the logs for full command output.

In this case, i already installed all the dependencies required MVC++, etc, but the problem continues.

I tried to reproduce it in another WIndows machine and it does not work, contrary, in a Linux based system it does work.

How to save the eif Model?

I am trying to save the model using pickle.dump() but this not working. How do I save the eif model?
Please provide me a solution as I am stuck with this problem. Thank you.

New release with fix for #14

#14 prevents installing the latest version of eif from pip.

This issue is fixed on master, but not on the latest release (from Nov 14): https://github.com/sahandha/eif/releases

@sahandha could you make a new release and push it to https://pypi.org/ ? I think that will solve a lot of the issues people are posting about in #14

Can the extension concept Applied to Gradient Boosted Machine?

Hi there,

This might be dummy questions.

I was curious whether the "extension" concept that you introduce can be applied to Supervised version such as Gradient Boosted Trees algorithm or not. There was several widely known Implementation like XGBoost or LightGBM. All of these GBT also suffer from "box" like decision boundary. I believe it would be great to see GBT to create decision boundary the way your extended isolation forest was producing.

What do you guys think?

Feel free to close this issue since its not real issue, just discussion.

Compile flags

Hi! I'm trying to install this using pip on Windows and I get compiler errors:

cl : Command line error D8021 : invalid numeric argument '/Wcpp'

It seems extra_compile_args in setup.py contains GCC-specific compiler arguments. Are these necessary? If not, can they be removed?

This issue has been addressed e.g. in the COCO API package for compatibility with non-GCC compilers:
https://github.com/philferriere/cocoapi/blob/master/PythonAPI/setup.py

Cant install eif on Py3.9

Hi there,

First I want to mention that I love the extended Isolation forest. Its a great algorithm and has yielded success for me personally more often then not. I am trying to install it in a python 3.9 environment and the pip installation fails. The full error is printed out below. I work on a ubuntu 21.04 machine and installed the environment with conda. This might be entirely unrelated, but I had a similar issue with the KDEpy library, and it seems as if there are issues with cython or the syntax changes of python 3.9.

If I can pro-actively support this library, I am more then happy to.

regards

(ug-16-04-2021) tv@tv-desktop:~$ pip install eif
Collecting eif
  Using cached eif-2.0.2.tar.gz (1.6 MB)
Requirement already satisfied: numpy in ./anaconda3/envs/ug-16-04-2021/lib/python3.9/site-packages (from eif) (1.20.2)
Requirement already satisfied: cython in ./anaconda3/envs/ug-16-04-2021/lib/python3.9/site-packages (from eif) (0.29.23)
Building wheels for collected packages: eif
  Building wheel for eif (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/tv/anaconda3/envs/ug-16-04-2021/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-o196o0b9/eif_7a59fee4c81c435c98e0405bce7f8a65/setup.py'"'"'; __file__='"'"'/tmp/pip-install-o196o0b9/eif_7a59fee4c81c435c98e0405bce7f8a65/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-0nvsces6
       cwd: /tmp/pip-install-o196o0b9/eif_7a59fee4c81c435c98e0405bce7f8a65/
  Complete output (24 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.9
  copying eif_old.py -> build/lib.linux-x86_64-3.9
  copying version.py -> build/lib.linux-x86_64-3.9
  running egg_info
  writing eif.egg-info/PKG-INFO
  writing dependency_links to eif.egg-info/dependency_links.txt
  writing requirements to eif.egg-info/requires.txt
  writing top-level names to eif.egg-info/top_level.txt
  reading manifest file 'eif.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching 'Readme.md'
  writing manifest file 'eif.egg-info/SOURCES.txt'
  running build_ext
  cythoning _eif.pyx to _eif.cpp
  building 'eif' extension
  creating build/temp.linux-x86_64-3.9
  gcc -pthread -B /home/tv/anaconda3/envs/ug-16-04-2021/compiler_compat -Wl,--sysroot=/ -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/tv/anaconda3/envs/ug-16-04-2021/include -fPIC -O2 -isystem /home/tv/anaconda3/envs/ug-16-04-2021/include -fPIC -I/home/tv/anaconda3/envs/ug-16-04-2021/lib/python3.9/site-packages/numpy/core/include -I/home/tv/anaconda3/envs/ug-16-04-2021/include/python3.9 -c _eif.cpp -o build/temp.linux-x86_64-3.9/_eif.o -Wcpp
  gcc: fatal error: cannot execute ‘cc1plus’: execvp: No such file or directory
  compilation terminated.
  error: command '/usr/bin/gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for eif
  Running setup.py clean for eif
Failed to build eif
Installing collected packages: eif
    Running setup.py install for eif ... error
    ERROR: Command errored out with exit status 1:
     command: /home/tv/anaconda3/envs/ug-16-04-2021/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-o196o0b9/eif_7a59fee4c81c435c98e0405bce7f8a65/setup.py'"'"'; __file__='"'"'/tmp/pip-install-o196o0b9/eif_7a59fee4c81c435c98e0405bce7f8a65/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-vgbpe2o1/install-record.txt --single-version-externally-managed --compile --install-headers /home/tv/anaconda3/envs/ug-16-04-2021/include/python3.9/eif
         cwd: /tmp/pip-install-o196o0b9/eif_7a59fee4c81c435c98e0405bce7f8a65/
    Complete output (24 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.9
    copying eif_old.py -> build/lib.linux-x86_64-3.9
    copying version.py -> build/lib.linux-x86_64-3.9
    running egg_info
    writing eif.egg-info/PKG-INFO
    writing dependency_links to eif.egg-info/dependency_links.txt
    writing requirements to eif.egg-info/requires.txt
    writing top-level names to eif.egg-info/top_level.txt
    reading manifest file 'eif.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'Readme.md'
    writing manifest file 'eif.egg-info/SOURCES.txt'
    running build_ext
    skipping '_eif.cpp' Cython extension (up-to-date)
    building 'eif' extension
    creating build/temp.linux-x86_64-3.9
    gcc -pthread -B /home/tv/anaconda3/envs/ug-16-04-2021/compiler_compat -Wl,--sysroot=/ -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/tv/anaconda3/envs/ug-16-04-2021/include -fPIC -O2 -isystem /home/tv/anaconda3/envs/ug-16-04-2021/include -fPIC -I/home/tv/anaconda3/envs/ug-16-04-2021/lib/python3.9/site-packages/numpy/core/include -I/home/tv/anaconda3/envs/ug-16-04-2021/include/python3.9 -c _eif.cpp -o build/temp.linux-x86_64-3.9/_eif.o -Wcpp
    gcc: fatal error: cannot execute ‘cc1plus’: execvp: No such file or directory
    compilation terminated.
    error: command '/usr/bin/gcc' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/tv/anaconda3/envs/ug-16-04-2021/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-o196o0b9/eif_7a59fee4c81c435c98e0405bce7f8a65/setup.py'"'"'; __file__='"'"'/tmp/pip-install-o196o0b9/eif_7a59fee4c81c435c98e0405bce7f8a65/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-vgbpe2o1/install-record.txt --single-version-externally-managed --compile --install-headers /home/tv/anaconda3/envs/ug-16-04-2021/include/python3.9/eif Check the logs for full command output.

Model saving

Hi,
is it possible to save a model, e.g. with pickle?
Thanks

Tree visualization problem +possible suggestion

Hi,

I was experimenting with this package for the last couple of days and I am liking it a lot!

I am at a point where I would like to visualize the trees or at least know the paths taken to isolate anomalies
but I couldn't figure out a way to see explicitly which features are used for exclusion.
The methods shown in EIF.ipynb don't seem to show explicitly the features and the ones in TreeVisualization.ipynb
didn't seem to work on the new version of eif.
Am I missing something or there actually is no way of knowing which features help finding anomalies?

Cheers!

Not an issue ... but is this available in R?

Not an issue, bur rather a question from one of the guys who is still fully bound to R. Any plans to provide this as a R package? Thanks, and congratulations for this work.

Difficulty in Parallelization

Hi,

I'm having trouble parallelizing the isolation forest algorithm for multiple sets of points using Python's multiprocessing.Pool . It seems to work as a single process. Could you let me know what are the changes I should make to the code to make this possible?

Thanks

module 'eif' has no attribute '__version__'

i install eif by "pip install eif" and Successfully installed eif-2.0.2
but when i use eif.iForest arise attributeError: module 'eif' has no attribute 'version'

Enhancement: Setting random state to the models

Hi,

First of all awesome job with the paper and library!

I believe a really nice (and relatively easy afaik) enhancement would be to implement random state as an input parameter for the iForest function, analogically to the one present at sklearn.

Cheers!

Tree visualizations page can not be loaded

Tree Vizaualizations page can not be loaded.

https://github.com/sahandha/eif/blob/master/Notebooks/TreeVisualization.ipynb

Add Arxiv paper to readme

Thanks for providing this code. Please add mention of and a link to your associated Arxiv paper into the repo's readme. The link is https://arxiv.org/abs/1811.02141

Error while installing eif

Hi!

Trying to install eif through pip I get the following error:

(base) C:\WINDOWS\system32>pip install eif
Collecting eif
  Using cached https://files.pythonhosted.org/packages/83/b2/d87d869deeb192ab599c899b91a9ad1d3775d04f5b7adcaf7ff6daa54c24/eif-2.0.2.tar.gz
Requirement already satisfied: numpy in c:\users\o.korshun\appdata\local\continuum\anaconda3\lib\site-packages (from eif) (1.16.5)
Requirement already satisfied: cython in c:\users\o.korshun\appdata\local\continuum\anaconda3\lib\site-packages (from eif) (0.29.13)
Building wheels for collected packages: eif
  Building wheel for eif (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\O0ADF~1.KOR\\AppData\\Local\\Temp\\pip-install-1adywqes\\eif\\setup.py'"'"'; __file__='"'"'C:\\Users\\O0ADF~1.KOR\\AppData\\Local\\Temp\\pip-install-1adywqes\\eif\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\O0ADF~1.KOR\AppData\Local\Temp\pip-wheel-kw_2kpwv' --python-tag cp37
       cwd: C:\Users\O0ADF~1.KOR\AppData\Local\Temp\pip-install-1adywqes\eif\
  Complete output (60 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-3.7
  copying eif_old.py -> build\lib.win-amd64-3.7
  copying version.py -> build\lib.win-amd64-3.7
  running egg_info
  writing eif.egg-info\PKG-INFO
  writing dependency_links to eif.egg-info\dependency_links.txt
  writing requirements to eif.egg-info\requires.txt
  writing top-level names to eif.egg-info\top_level.txt
  reading manifest file 'eif.egg-info\SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'eif.egg-info\SOURCES.txt'
  running build_ext
  cythoning _eif.pyx to _eif.cpp
  building 'eif' extension
  creating build\temp.win-amd64-3.7
  creating build\temp.win-amd64-3.7\Release
  C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\Library\mingw-w64\bin\gcc.exe -mdll -O -Wall -DMS_WIN64 -IC:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include -IC:\Users\o.korshun\AppData\Local\Continuum\anaconda3\include -IC:\Users\o.korshun\AppData\Local\Continuum\anaconda3\include -c _eif.cpp -o build\temp.win-amd64-3.7\Release\_eif.o -Wcpp
  In file included from C:/Users/o.korshun/AppData/Local/Continuum/anaconda3/Library/mingw-w64/include/c++/5.3.0/random:35:0,
                   from eif.hxx:5,
                   from _eif.cpp:614:
  C:/Users/o.korshun/AppData/Local/Continuum/anaconda3/Library/mingw-w64/include/c++/5.3.0/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
   #error This file requires compiler and library support for the \
    ^
  In file included from C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include/numpy/ndarraytypes.h:1822:0,
                   from C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include/numpy/ndarrayobject.h:12,
                   from C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include/numpy/arrayobject.h:4,
                   from _eif.cpp:612:
  C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h:15:77: note: #pragma message: C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h(14) : Warning Msg: Using deprecated NumPy API, disable it with #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
                            "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION")
                                                                               ^
  In file included from _eif.cpp:614:0:
  eif.hxx:11:28: error: 'std::mt19937_64' has not been declared
   #define RANDOM_ENGINE std::mt19937_64
                              ^
  eif.hxx:65:55: note: in expansion of macro 'RANDOM_ENGINE'
           void build_tree (double*, int, int, int, int, RANDOM_ENGINE&, int);
                                                         ^
  eif.hxx:11:28: error: 'std::mt19937_64' has not been declared
   #define RANDOM_ENGINE std::mt19937_64
                              ^
  eif.hxx:66:44: note: in expansion of macro 'RANDOM_ENGINE'
           Node* add_node (double*, int, int, RANDOM_ENGINE&);
                                              ^
  eif.hxx:11:28: error: 'std::mt19937_64' has not been declared
   #define RANDOM_ENGINE std::mt19937_64
                              ^
  eif.hxx:132:63: note: in expansion of macro 'RANDOM_ENGINE'
   inline std::vector<int> sample_without_replacement (int, int, RANDOM_ENGINE&);
                                                                 ^
  _eif.cpp: In function 'PyTypeObject* __Pyx_ImportType(PyObject*, const char*, const char*, size_t, __Pyx_ImportType_CheckSize)':
  _eif.cpp:8085:53: warning: unknown conversion type character 'z' in format [-Wformat=]
               module_name, class_name, size, basicsize);
                                                       ^
  _eif.cpp:8085:53: warning: unknown conversion type character 'z' in format [-Wformat=]
  _eif.cpp:8085:53: warning: too many arguments for format [-Wformat-extra-args]
  error: command 'C:\\Users\\o.korshun\\AppData\\Local\\Continuum\\anaconda3\\Library\\mingw-w64\\bin\\gcc.exe' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for eif
  Running setup.py clean for eif
Failed to build eif
Installing collected packages: eif
    Running setup.py install for eif ... error
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\O0ADF~1.KOR\\AppData\\Local\\Temp\\pip-install-1adywqes\\eif\\setup.py'"'"'; __file__='"'"'C:\\Users\\O0ADF~1.KOR\\AppData\\Local\\Temp\\pip-install-1adywqes\\eif\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\O0ADF~1.KOR\AppData\Local\Temp\pip-record-yqa9lmac\install-record.txt' --single-version-externally-managed --compile
         cwd: C:\Users\O0ADF~1.KOR\AppData\Local\Temp\pip-install-1adywqes\eif\
    Complete output (60 lines):
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.7
    copying eif_old.py -> build\lib.win-amd64-3.7
    copying version.py -> build\lib.win-amd64-3.7
    running egg_info
    writing eif.egg-info\PKG-INFO
    writing dependency_links to eif.egg-info\dependency_links.txt
    writing requirements to eif.egg-info\requires.txt
    writing top-level names to eif.egg-info\top_level.txt
    reading manifest file 'eif.egg-info\SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'eif.egg-info\SOURCES.txt'
    running build_ext
    skipping '_eif.cpp' Cython extension (up-to-date)
    building 'eif' extension
    creating build\temp.win-amd64-3.7
    creating build\temp.win-amd64-3.7\Release
    C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\Library\mingw-w64\bin\gcc.exe -mdll -O -Wall -DMS_WIN64 -IC:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include -IC:\Users\o.korshun\AppData\Local\Continuum\anaconda3\include -IC:\Users\o.korshun\AppData\Local\Continuum\anaconda3\include -c _eif.cpp -o build\temp.win-amd64-3.7\Release\_eif.o -Wcpp
    In file included from C:/Users/o.korshun/AppData/Local/Continuum/anaconda3/Library/mingw-w64/include/c++/5.3.0/random:35:0,
                     from eif.hxx:5,
                     from _eif.cpp:614:
    C:/Users/o.korshun/AppData/Local/Continuum/anaconda3/Library/mingw-w64/include/c++/5.3.0/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
     #error This file requires compiler and library support for the \
      ^
    In file included from C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include/numpy/ndarraytypes.h:1822:0,
                     from C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include/numpy/ndarrayobject.h:12,
                     from C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include/numpy/arrayobject.h:4,
                     from _eif.cpp:612:
    C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h:15:77: note: #pragma message: C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h(14) : Warning Msg: Using deprecated NumPy API, disable it with #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
                              "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION")
                                                                                 ^
    In file included from _eif.cpp:614:0:
    eif.hxx:11:28: error: 'std::mt19937_64' has not been declared
     #define RANDOM_ENGINE std::mt19937_64
                                ^
    eif.hxx:65:55: note: in expansion of macro 'RANDOM_ENGINE'
             void build_tree (double*, int, int, int, int, RANDOM_ENGINE&, int);
                                                           ^
    eif.hxx:11:28: error: 'std::mt19937_64' has not been declared
     #define RANDOM_ENGINE std::mt19937_64
                                ^
    eif.hxx:66:44: note: in expansion of macro 'RANDOM_ENGINE'
             Node* add_node (double*, int, int, RANDOM_ENGINE&);
                                                ^
    eif.hxx:11:28: error: 'std::mt19937_64' has not been declared
     #define RANDOM_ENGINE std::mt19937_64
                                ^
    eif.hxx:132:63: note: in expansion of macro 'RANDOM_ENGINE'
     inline std::vector<int> sample_without_replacement (int, int, RANDOM_ENGINE&);
                                                                   ^
    _eif.cpp: In function 'PyTypeObject* __Pyx_ImportType(PyObject*, const char*, const char*, size_t, __Pyx_ImportType_CheckSize)':
    _eif.cpp:8085:53: warning: unknown conversion type character 'z' in format [-Wformat=]
                 module_name, class_name, size, basicsize);
                                                         ^
    _eif.cpp:8085:53: warning: unknown conversion type character 'z' in format [-Wformat=]
    _eif.cpp:8085:53: warning: too many arguments for format [-Wformat-extra-args]
    error: command 'C:\\Users\\o.korshun\\AppData\\Local\\Continuum\\anaconda3\\Library\\mingw-w64\\bin\\gcc.exe' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Users\o.korshun\AppData\Local\Continuum\anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\O0ADF~1.KOR\\AppData\\Local\\Temp\\pip-install-1adywqes\\eif\\setup.py'"'"'; __file__='"'"'C:\\Users\\O0ADF~1.KOR\\AppData\\Local\\Temp\\pip-install-1adywqes\\eif\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\O0ADF~1.KOR\AppData\Local\Temp\pip-record-yqa9lmac\install-record.txt' --single-version-externally-managed --compile Check the logs for full command output.

Consider extending iForest for categorical/mixed data

Would you consider extending the implementation of Extended IsolationForest for categorical/mixed data using the approach described in https://www.sciencedirect.com/science/article/pii/S1877050918311852 ?

PR for Parallelization and Reduce Memory

Hello,

For high dimensional datasets, I'm finding multi-processing parallelization can speed things up a bit. I also, find that storing the original data in each Node and each iTree consumes a lot of needless memory. Would you be open to reviewing a Pull Request(s) that addressed both of these items? If so, would you accept them bundled together as one PR or would you like them separated?

Thanks

Unable to install eif2.0.2

Dear Team,
I am getting below error while trying install eif2.02 .
Methods tried:

1. pip install eif
2. Downloaded eif tar file from pypi.org and tried installing
3. Downloaded the code from github and tried installing
4. In one of the issue it is mentioned to edit setup.py file(Remove the extra_compile line) and executed

failed in all above methods,
Below is the error
ERROR: Complete output from command 'C:\Anaconda3\python.exe' -u -c 'import setuptools, tokenize;file='"'"'C:\Users\XSVIJA1\AppData\Local\Temp\pip-req-build-rqacf45o\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\XSVIJA1\AppData\Local\Temp\pip-wheel-wjzxwp64' --python-tag cp37:
ERROR: running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.7
copying eif_old.py -> build\lib.win-amd64-3.7
copying version.py -> build\lib.win-amd64-3.7
running egg_info
writing eif.egg-info\PKG-INFO
writing dependency_links to eif.egg-info\dependency_links.txt
writing requirements to eif.egg-info\requires.txt
writing top-level names to eif.egg-info\top_level.txt
reading manifest file 'eif.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'eif.egg-info\SOURCES.txt'
running build_ext
cythoning _eif.pyx to _eif.cpp
building 'eif' extension
creating build\temp.win-amd64-3.7
creating build\temp.win-amd64-3.7\Release
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Anaconda3\lib\site-packages\numpy\core\include -IC:\Anaconda3\include -IC:\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt" /EHsc /Tp_eif.cpp /Fobuild\temp.win-amd64-3.7\Release_eif.obj -Wcpp
cl : Command line error D8021 : invalid numeric argument '/Wcpp'
error: command 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe' failed with exit status 2

ERROR: Failed building wheel for eif
Running setup.py clean for eif
Failed to build eif
Installing collected packages: eif
Running setup.py install for eif ... error
ERROR: Complete output from command 'C:\Anaconda3\python.exe' -u -c 'import setuptools, tokenize;file='"'"'C:\Users\XSVIJA1\AppData\Local\Temp\pip-req-build-rqacf45o\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\XSVIJA1\AppData\Local\Temp\pip-record-f8wv7_fl\install-record.txt' --single-version-externally-managed --compile:
ERROR: running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.7
copying eif_old.py -> build\lib.win-amd64-3.7
copying version.py -> build\lib.win-amd64-3.7
running egg_info
writing eif.egg-info\PKG-INFO
writing dependency_links to eif.egg-info\dependency_links.txt
writing requirements to eif.egg-info\requires.txt
writing top-level names to eif.egg-info\top_level.txt
reading manifest file 'eif.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'eif.egg-info\SOURCES.txt'
running build_ext
skipping '_eif.cpp' Cython extension (up-to-date)
building 'eif' extension
creating build\temp.win-amd64-3.7
creating build\temp.win-amd64-3.7\Release
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Anaconda3\lib\site-packages\numpy\core\include -IC:\Anaconda3\include -IC:\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt" /EHsc /Tp_eif.cpp /Fobuild\temp.win-amd64-3.7\Release_eif.obj -Wcpp
cl : Command line error D8021 : invalid numeric argument '/Wcpp'
error: command 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe' failed with exit status 2
----------------------------------------
ERROR: Command "'C:\Anaconda3\python.exe' -u -c 'import setuptools, tokenize;file='"'"'C:\Users\XSVIJA1\AppData\Local\Temp\pip-req-build-rqacf45o\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\XSVIJA1\AppData\Local\Temp\pip-record-f8wv7_fl\install-record.txt' --single-version-externally-managed --compile" failed with error code 1 in C:\Users\XSVIJA~1\AppData\Local\Temp\pip-req-build-rqacf45o\

Please help.

Scoring takes too long

My training and validation data are of similar size (about 1,500,000 rows and 11 features). Model building took very less time even with full extension. But, when scoring the validation data using compute_paths, the function has been running for close to 15 hours and still scoring is not done. Is there some way to speed up the scoring process?

I can't install eif 2.0.2, please tell me the reason

(base) C:\Users\22393\eif-2.0.2\eif-2.0.2>python setup.py install
running install
running bdist_egg
running egg_info
writing eif.egg-info\PKG-INFO
writing dependency_links to eif.egg-info\dependency_links.txt
writing requirements to eif.egg-info\requires.txt
writing top-level names to eif.egg-info\top_level.txt
reading manifest file 'eif.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'eif.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_py
running build_ext
skipping '_eif.cpp' Cython extension (up-to-date)
building 'eif' extension
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IE:\ProgramFiles\anaconda\lib\site-packages\numpy\core\include -IE:\ProgramFiles\anaconda\include -IE:\ProgramFiles\anaconda\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt" /EHsc /Tp_eif.cpp /Fobuild\temp.win-amd64-3.8\Release_eif.obj -Wcpp
cl: 命令行 error D8021 :无效的数值参数“/Wcpp”
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe' failed with exit status 2

Effect of feature scaling

Hi thanks for the great package (and example notebooks!). My issue is summarised in two points:

• It appears that feature scale influences the orientation of the hyperplane splits in the trees, resulting in a poor anomaly score map.
• Is this expected behaviour? If so, can anyone offer an explanation as to how this comes about as it seems from the paper that the orientation of all hyperplanes are random.

The following illustrates this further:

I have noticed that the extended forest shows odd results when applied to features with very different scales. For example if I draw 2D points from 2 normal distributions with variance 1 and 1000 and plot the contour maps comparing the regular iForest and the extended we see the contours become horizontal and the heat map in general is not good compared to the regular iForest.

It seems as though the choice of hyperplane gets biased towards horizontal lines. This is also notable in the examples given in the paper (figure 9) where 3 plots of tree splits are shown:

Here we see the first two examples (a and b) the x and y values of the data lie on the same scale and the splits look randomly orientated. However in c) the x scale of the data is much larger than y scale, and most splits look more vertical. As a result we seen areas of higher anomaly score above and below the point cloud in the resulting heat map:

This issue is easily fixed by simply scaling all features before using the forest. However I was wondering if the splits are done on a hyperplane of random orientation why/how does feature scale influence the orientation of splits in each tree?

Apologies if I am missing something obvious, any insight would be useful, thanks!

Use in novelty detection/one-class classification

From what I understand, your api doesn't distinguish between constructing the trees and querying to obtain scores (like the fit/predict methods of scikit-learn), is that correct?

So it's not currently possible to use this implementation for novelty detection/one-class classification, where the training set is different from the test set?

setting ExtensionLevel

If I understand the paper correctly, we obtain the full EIF approach by setting ExtensionLevel equal to the number of dimensions of the data minus 1, correct?

No an issue but does sklearn's GridSearchCV or RandomSearchCV work with this?

No an issue but does sklearn's GridSearchCV or RandomSearchCV work with this?

It would be great to use it if possible.

Cheers

Docstrings and method signatures

When running in Jupyter or ipython, attempting to bring up the documentation for the iForest class produces only:

Init signature: iso.iForest(self, /, *args, **kwargs)
Docstring:      <no docstring>

While the example notebooks provide sufficient guidance on how to use the iForest class, it would be useful to have this available at the point of usage - both the full argument specification and the docstrings.

This might just be a question of setting the binding compile option of Cython to true (e.g. via decorators @cython.binding(True).

Happy to help out and create a PR if this would be desired/useful.