donnemartin/data-science-ipython-notebooks

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

data-science-ipython-notebooks's Issues

README is missing contributing section

Request for sage references

Title says it all.

Is it possible to include a section for Sage related notebooks?

Some of the links are giving 404 error

I tried rnn-lstm in Keras, the link seems to be expired.

There are other many links too showing 404 error. Please fix them.

Python 3 compatibly issues

flake8 testing of https://github.com/donnemartin/data-science-ipython-notebooks on Python 3.7.0

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./scipy/thinkplot.py:475:14: F821 undefined name 'xp'
        xs = xp.delete(xs, 0)
             ^
./data/titanic/myfirstforest.py:84:19: E999 SyntaxError: invalid syntax
print 'Training...'
                  ^
./data/titanic/gendermodel.py:47:46: E999 SyntaxError: invalid syntax
print 'Proportion of women who survived is %s' % proportion_women_survived
                                             ^
./data/titanic/genderclassmodel.py:42:10: F821 undefined name 'xrange'
for i in xrange(number_of_classes):
         ^
./data/titanic/genderclassmodel.py:43:14: F821 undefined name 'xrange'
    for j in xrange(number_of_price_brackets):
             ^
./data/titanic/genderclassmodel.py:84:14: F821 undefined name 'xrange'
    for j in xrange(number_of_price_brackets):
             ^
./deep-learning/keras-tutorial/deep_learning_models/vgg19.py:163:9: F821 undefined name 'preprocess_input'
    x = preprocess_input(x)
        ^
./deep-learning/keras-tutorial/deep_learning_models/vgg19.py:167:25: F821 undefined name 'decode_predictions'
    print('Predicted:', decode_predictions(preds))
                        ^
./deep-learning/keras-tutorial/deep_learning_models/resnet50.py:242:9: F821 undefined name 'preprocess_input'
    x = preprocess_input(x)
        ^
./deep-learning/keras-tutorial/deep_learning_models/resnet50.py:246:25: F821 undefined name 'decode_predictions'
    print('Predicted:', decode_predictions(preds))
                        ^
./deep-learning/keras-tutorial/deep_learning_models/vgg16.py:161:9: F821 undefined name 'preprocess_input'
    x = preprocess_input(x)
        ^
./deep-learning/keras-tutorial/deep_learning_models/vgg16.py:165:25: F821 undefined name 'decode_predictions'
    print('Predicted:', decode_predictions(preds))
                        ^
./deep-learning/keras-tutorial/solutions/sol_112.py:2:1: E999 SyntaxError: invalid syntax
%timeit -n 1 -r 1 ann.train(zip(X,y), iterations=100)
^
./deep-learning/keras-tutorial/solutions/sol_111.py:2:1: E999 SyntaxError: invalid syntax
%timeit -n 1 -r 1 ann.train(zip(X,y), iterations=2)
^
./deep-learning/theano-tutorial/rnn_tutorial/lstm_text.py:255:22: E999 SyntaxError: invalid syntax
        print 'epoch:', epoch
                     ^
./deep-learning/tensor-flow-examples/multigpu_basics.py:84:37: E999 SyntaxError: invalid syntax
print "Single GPU computation time: " + str(t2_1-t1_1)
                                    ^
./deep-learning/tensor-flow-examples/input_data.py:95:34: F821 undefined name 'xrange'
      fake_image = [1.0 for _ in xrange(784)]
                                 ^
./deep-learning/tensor-flow-examples/input_data.py:97:35: F821 undefined name 'xrange'
      return [fake_image for _ in xrange(batch_size)], [
                                  ^
./deep-learning/tensor-flow-examples/input_data.py:98:31: F821 undefined name 'xrange'
          fake_label for _ in xrange(batch_size)]
                              ^
6     E999 SyntaxError: invalid syntax
13    F821 undefined name 'xrange'
19

Add requirements file to help with installation for users who prefer not to use Anaconda

sddd

fifofoof

Migrate notebook references from the now deprecated EMR CLI to the recommended AWS CLI

Update the CNN notebook using the latest TensorFlow version

I want to update this notebook using the latest TensorFlow version 2.4.1.

Please assign this issue to me.

solving issue

data-science-ipython-notebooks

Add notebook for Bokeh

"Bokeh is a Python interactive visualization library that targets modern web browsers for presentation. Its goal is to provide elegant, concise construction of novel graphics in the style of D3.js, but also deliver this capability with high-performance interactivity over very large or streaming datasets. Bokeh can help anyone who would like to quickly and easily create interactive plots, dashboards, and data applications."

Bokeh seems like a good candidate to feed data from Spark streaming and sharing results to stakeholders who don't use visualization tools like Tableau.

Bokeh at Pycon

PyTorch tutorials

Hey, I see that there are no tutorial notebooks for implementing machine learning algorithms and neural networks in PyTorch in this repo yet. PyTorch is gaining a lot of traction lately and is really going to be one of the most popular frameworks due to its dynamic computational graph & eager execution.

I would like to add such tutorial notebooks in PyTorch.

Data Science

Add deep learning notebooks

Trax Tutorials

Hey,

I see that there are no tutorial notebooks for Trax implementations in this repository yet. Trax is an end-to-end library for deep learning that focuses on clear code and speed. It is actively used and maintained in the Google Brain team.

I would like to add such tutorial notebooks in Trax

Ipython notebook

Add statistical inference notebooks showing scipy

Allen Downey's workshop on statistical inference with computation methods in Python was a great tutorial during Pycon 2015.

https://github.com/AllenDowney/CompStats

Add notebook for AWS CLI setup and common commands

Churn1

Machine learning models

Machine learning models with solved datasets and solutions

Data preprocessing

I'm trying to clean my data and do some preprocessing but i don't have much understanding of the Columns if the zero in them are normal or missing values i'm using the dataset cic-collection on kaggle if any expert would help i'd be much thankful

matplotlib-applied.ipynb imports issues

The split of this notebook from matplotlib.ipynb left out key imports, which is now causing errors when running the notebook.

Disco Project

Why don't menction also Disco Project (http://discoproject.org/)?
It's an easy Map Reduce framework in Python (jobs) + Erlang (scheduling) by Nokia

Update Spark notebook to use DataFrames

Spark 1.3 introduced DataFrames:

A DataFrame is a distributed collection of data organized into named columns. It is conceptually equivalent to a table in a relational database or a data frame in R/Python, but with richer optimizations under the hood. DataFrames can be constructed from a wide array of sources such as: structured data files, tables in Hive, external databases, or existing RDDs.

https://spark.apache.org/docs/1.3.0/sql-programming-guide.html#dataframes

Spark 1.6 introduces DataSets:

A new Dataset API. This API is an extension of Spark’s DataFrame API that supports static typing and user functions that run directly on existing JVM types (such as user classes, Avro schemas, etc). Dataset’s use the same runtime optimizer as DataFrames, but support compile-time type checking and offer better performance. More details on Datasets can be found on the SPARK-9999 JIRA.

https://databricks.com/blog/2015/11/20/announcing-spark-1-6-preview-in-databricks.html

Note: PySpark will included DataSets in the upcoming Spark 2.0.

Command to run mrjob s3 log parser is incorrect

Current:

python mr-mr_s3_log_parser.py -r emr s3://bucket-source/ --output-dir=s3://bucket-dest/"

Should be:

python mr_s3_log_parser.py -r emr s3://bucket-source/ --output-dir=s3://bucket-dest/"

Add Keras notebooks

.

New project

https://github.com/facebookresearch/dynalab/blob/main/.gitignore#L1-L27

"Error 503 No healthy backends"

Hello,

When I try to open the hyperlinks which should direct me to the correct ipython notebook, it returns me "Error 503 No healthy backends"
"No healthy backends
Guru Mediation:
Details: cache-fra1236-FRA 1462794681 3780339426
Varnish cache server"

Thanks
Jiahong Wang

Add simplified Spark installation instructions from the repo: https://github.com/donnemartin/dev-setup

Mac users can benefit from a much simplified installation method thanks to Homebrew.

Add SAWS: A Supercharged AWS Command Line Interface (CLI) to AWS Section.

code

sir ,plz send me code along with churn dataset

Python

pandas - Working with Strings

In the code cell 3:
data = ['peter', 'Paul', None, 'MARY', 'gUIDO']
data.map(lambda s: s.capitalize())
was changed to
[s.capitalize() for s in data if s!= None ]
and now the code woks fine.

Add instructions to configure IPython/PySpark for python 3, now supported with Spark 1.4

Reported by core_dumpd on Reddit /r/DataScience.

Solution seems to be discussed in Stack Overflow here.

core_dumpd reports the following works, need to confirm and update repo:

I end up running this:
PYSPARK_DRIVER_PYTHON_OPTS="notebook --profile=pyspark" /usr/local/spark/bin/pyspark

With:
PYSPARK_PYTHON=/opt/anaconda/bin/ipython PYSPARK_DRIVER_PYTHON=/opt/anaconda/bin/ipython

I'm running on docker based on sequenceiq/hadoop-docker:latest with Spark/MiniConda added on top. The only real config options in the profile are for the ip = '*' and open_browser = False.

Add notebook info for Boto, the official AWS SDK for Python.

alexnet.ipynb contains incomplete architecture of alexnet(2 cnn layers missing)

Alexnet implementation in tensorflow has incomplete architecture where 2 convolution neural layers are missing. This issue is in reference to the python notebook mentioned below.

https://github.com/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/notebooks/3_neural_networks/alexnet.ipynb

solve issue

data science learning is difficult.

Data science

Notice: Maintainer Temporarily Unavailable

Hi Everyone,

I'll be temporarily unavailable starting April 17 and won't be able to respond to issues or pull requests. I'm hoping I'll be available in a couple of weeks, but I might not be responsive for several weeks.

Sorry for the inconvenience.

-Donne