aidelabaz/weather_and_agriculture

AIDE Lab repository for LSMS-ISA and remotely sensed weather data cleaning code.

Weather and Agriculture: Cleaning code for LSMS-ISA and remotely sensed weather data integration project

This repository acts as the mother ship for a number of other repositories that hold replication code for papers sprining from the main project. The main project site is on OSF and has the goal of exploring a variety of issues that arise when remotely sensed weather data is integrated with socioeconomic survey data. The dependency or subsidiary repos from this repo include replication packages for:

• Michler, J.D., Josephson, A., Kilic, T., and Murray, S. (2021). "Estimating the Impact of Weather on Agriculture." World Bank Policy Research Working Paper, No. 9867.
• Michler, J.D., Josephson, A., Kilic, T., and Murray, S. (2022). "Privacy Protection, Measurement Error, and the Integration of Remote Sensing and Socioeconomic Survey Data." Journal of Development Economics 158: 102927.
• Josephson, A., Michler, J.D., Kilic, T., and Murray, S. (2024). "The Mismeasure of Weather: Using Remotely Sensed Weather in Economic Contexts."
• Agme, C., Josephson, A., Michler, J.D., Kilic, T., and Murray, S. (2024). "Variable Selection in Economic Applications of Remotely Sensed Weather Data: Evidence from the LSMS-ISA."

Because the weather data contains confidential information, it is not publically available. This means the weather code will not function, as that data is held by the World Bank. Contact Drs. Jeffrey D. Michler or Anna Josephson and they can share an intermediate - de-identified - version of the weather data for use in replicating the results.

This README was last updated on 3 September 2024.

Index

• Project Team
• Data cleaning
• Pre-requisites
• Folder structure

Project Team

Contributors:

• Jeffrey D. Michler [[email protected]] (Conceptualizaiton, Supervision, Visualization, Writing)
• Anna Josephson [[email protected]] (Conceptualizaiton, Supervision, Visualization, Writing)
• Talip Kilic (Conceptualization, Resources, Writing)
• Siobhan Murray (Conceptualization, Writing)
• Brian McGreal (Data curation)
• Alison Conley (Data curation)
• Emil Kee-Tui (Data curation)
• Reece Branham (Data curation)
• Rodrigo Guerra Su (Data curation)
• Jacob Taylor (Data curation)
• Kieran Douglas (Data curation)

Data cleaning

The code in this repository is primarily for replicating the cleaning of the household LSMS-ISA data. This requires downloading this repo and the household data from the World Bank webiste. The projectdo.do should then replicate the data cleaning process.

Pre-requisites

Stata req's

• The data processing and analysis requires a number of user-written Stata programs:
  1. weather_command
  2. blindschemes
  3. estout
  4. winsor2
  5. mdesc
  6. distinct

Folder structure

The OSF project page provides more details on the data cleaning.

For the household cleaning code to run, the public use microdata must be downloaded from the World Bank Microdata Library. Furthermore, the data needs to be placed in the following folder structure:

weather_and_agriculture
├────household_data      
│    └──country          /* one dir for each country */
│       ├──wave          /* one dir for each wave */
│       └──logs
├──weather_data
│    └──country          /* one dir for each country */
│       ├──wave          /* one dir for each wave */
│       └──logs
├──merged_data
│    └──country          /* one dir for each country */
│       ├──wave          /* one dir for each wave */
│       └──logs
├──regression_data
│    ├──country          /* one dir for each country */
│    └──logs
└────results_data        /* overall analysis */
     ├──tables
     ├──figures
     └──logs

weather_and_agriculture's Issues

labor use (days/ha)

2019_agsec3a household count max values are greater than what people were allowed to report

Describe the bug
Working in 2019_agsec3a and noticed that the household count values seemed odd. There are a ton of 0s and the max values exceed the 1-5 that people were allowed to report. This is in section 5 Labor.

Kieran general issue with logs

Kieran having trouble with log files. @KieranCDouglas - could you add a note here which describes the error you see?

land area - all crops (ha)

land area - all crops (ha)

land area - all crops (ha)

merge each household data set into one wave data set

production - primary crop (kg)

labor use (days/kg)

fertilizer application rate (kg/ha)

production - primary crop (kg)

labor use (days/kg)

merge each household data set into one wave data set

yield - primary (kg/ha)

total value of production - all crops (local currency)

In Uganda we generate crop value in two ways, one imputing value based on sold amount and the other calculating local prices and imputing values based on prices times quantity. The first one is what we used in Uganda while the other method is used in other countries. The first method results in lower crop values. Based on summary stats in the populated PAP Uganda has much lower yields and much lower crop value than other countries. Once all waves are completed we should reassess the decision about which value imputation method to use.

Wrong data file is opened when I import 2019_agsec5a

When I attempt to import 2019_agsec5a, it works but the data it opens is from 2014/2015 rather than 2019.

MWI conversion to 2015 - need to find conversion in original code

not sure where it is, but this will be a pain, want to brainstorm ideas beyond just opening every file.

Almost 2/3 of the values in sold data are 0 in 2019_agsec5a

Describe the bug
Line 305 under section 4 (generating sold harvest values) attemps to replace zeros in sold data as missing. When you run it, ~2/3 of the values are then converted to missing (3932).

To Reproduce
Steps to reproduce the behavior:

1. Go to line 304 in 2019_agsec5a
2. Run line: replace s5aq07a_1 = . if s5aq07a_1 == 0
3. Converts 3932 to missing

Screenshots