usgs-r / ds-gitflows-collnell Goto Github PK
View Code? Open in Web Editor NEWColleen's repository for learning and practicing the Data Science Git/GitHub workflow.
Colleen's repository for learning and practicing the Data Science Git/GitHub workflow.
In the last issue, we pulled down changes from our collaborator and realized that there was a merge conflict. This occurs when you edit the same line of code. In this case, you both added on to the last line of the file. Don't worry - merge conflicts are not as intimidating as they may seem. Let's do this!
Action: Fix the merge conflict.
dryville_story.md
file.<<<<<<<
at the start and >>>>>>>
at the end. Sometimes, there are multiple conflicts and each will be sectioned off with the left and right pointing carrots.dryville_story.md
. The content between <<<<<<< HEAD
and =======
is the content that existed in your version of the file before you attempted to pull other changes. The content between =======
and >>>>>>>
is what you pulled down and tried to merge. Examine the differences between the content in those sections.Your First Flood
section, while the version you are trying to merge does not. So, you would want to keep the Your First Flood
text. Cut and paste that section so that it is outside and above the <<<<<<< HEAD
section (see below).## Your First Flood
You're again happy until the first desert downpour hits. The rain flows down the hills (runoff) into Dryville's town center and suddenly you have your first flood — more unwanted water (and the mud it carries with it) to deal with. You decide to build a set of storm drains to fix this problem. Lay some more (this time BIG) pipes through town with intakes where the water collects in low spots. Storm water will flow into these pipes and be sent on its way downhill into your creek. Another problem solved.
But when the storm hit, Dryville Creek overflowed and flooded some houses that were built on the flood plain, the flat ground alongside of the creek. You can do two things here. Look at the lay of the land and decide what parts of the creek bed will flood most often when it really rains and don't allow people to build houses there, or build a dam upstream to create a reservoir to trap storm water before it floods into town. Your reservoir can then release the water slowly over a long period of time, thus preventing floods and recharging ground water.
<<<<<<< HEAD
## Storing Water for a Rainy Day
You start thinking... a reservoir (you can call it a lake) above town could really serve a lot of purposes. A lake will provide a place for you to have fun — go swimming, boating, catch catfish, and relax. You can run your water-supply intake pipes from the lake instead of from your creek, especially since the flood destroyed your water-intake pumping station. With a dam you can release only the amount of water you want into the creek below the dam, thus making sure you have just the right amount of water running in Dryville Creek at all times. A dam would even help prevent flooding downstream because you can hold extra rainfall and runoff during a storm and slowly release it afterward. You can build a bigger paddle wheel, or, better yet, construct a real [hydroelectric power plant](https://www.usgs.gov/special-topic/water-science-school/science/hydroelectric-power-water-use) in your dam to start generating electricity! More problems solved.
=======
## Storing Water for a Rainy Day
You start thinking... a reservoir (you can call it a lake) above town could really serve a lot of purposes. A lake will provide a place for you to have fun — go swimming, boating, catch catfish, and relax. You can run your water-supply intake pipes from the lake instead of from your creek, especially since the flood destroyed your water-intake pumping station. With a dam you can release only the amount of water you want into the creek below the dam, thus making sure you have just the right amount of water running in Dryville Creek at all times. A dam would even help prevent flooding downstream because you can hold extra rainfall and runoff during a storm and slowly release it afterward. You can build a bigger paddle wheel, or, better yet, construct a real hydroelectric power plant in your dam to start generating electricity! More problems solved.
>>>>>>> afd002e4b66f31f815e4236ffa6ea3e17f127e1d
=======
and >>>>>>>
.<<<<<<< HEAD
, =======
, and >>>>>>> [random letters/numbers]
). Now, you should be back to where you started (which happens in merge conflict resolution sometimes).git status
will show you that you have an unresolved merge conflict. Just as before, run git add dryville_story.md
to stage your changed file. Then commit by running git commit -m "resolve merge conflict"
.git status
you will see that you don't have any changes to commit (but your branch still ahead of origin/master - see next issue).You successfully resolved a conflict! Close this issue and move on to the next one.
Scenario: You are pointed to a code repository on GitHub (this one) for a project that you just joined. You need to start contributing to this codebase. Where do you start?
So far in the GitHub lessons, you've learned about using GitHub for version control to make and save changes (commit) and merge those changes into the code (pull request). You were also taught how to use branches to keep your code separate from the canonical codebase. However, that is not the only workflow that can be followed when using Git and GitHub. In our group, we use a slightly different version in order to manage collaborative development. Below is a generic depiction of our version control workflow. It is known more widely as the "fork-and-pull workflow".
There is a main version of the code that people are collaboratively developing. Each contributor has their own version of this code online and locally. Changes are made locally, sent to their online version, and then combined with the collaborative version of the code. Contributors are able to get the changes from other users by syncing their local version with the collaborative version of the code. Now let's look at that workflow using Git + GitHub terminology.
There is a main version of the code that people are collaboratively developing (upstream repository
). Each contributor has their own version of this code online (forked repository
) and locally (cloned repository
). Changes are saved locally (commit
), sent to their online version (pushed to their fork
), and then combined with the collaborative version of the code (merged with a pull request
). Contributors are able to get the changes from other users by syncing their local version with the collaborative version of the code (pull the upstream repository
).
We are going to walk through each of these steps within the workflow in this lesson. We will also learn about merge conflicts and what a .gitignore
file is all about.
Action: Close this issue after you read about the workflow and proceed to the next issue in sequential order.
At this point, we have made our file changes and are satisfied with the state of our local repository. It is time to join these changes with the main repository so that our collaborators can use them. Before we can merge our changes with the main repository, we need to get our local changes onto our fork on GitHub.
Action: Push your changes to your GitHub fork.
git status
in Git Bash right now, you should see a message that says "nothing to commit" and also "Your branch is ahead of 'origin/master' by 2 commits". This means that our local version has 2 new changes that do not appear in our remote fork (called the origin
because our local version originated from it).git push
. By default, it will push changes up to the origin
remote master
branch (we aren't using branching just yet, so everything is the master
branch). If you want to be more explicit, you can run git push [remote name] [branch name]
. For example, git push origin master
will do the same as git push
.https://github.com/[username]/ds-gitflows-[username]
) and click on the commits
button (see image below). You should see your two commit messages appear there.You have pushed your changes up to GitHub! Close this issue and move on to the next one.
The next step in our workflow is to clone your fork. This creates a local copy of the repository that is specific to your user on GitHub. The local copy is where you will make changes to the codebase.
Action: Clone this repo!
https://github.com/[username]/ds-gitflows-[username]
. A navigation note: from your fork, you can easily navigate back to the canonical repository by clicking the link next to "forked from" at the top, just below your forked repository name. From the canonical repo page, you can get back to your fork by clicking the fork button on the canonical repo and choosing your existing fork from the list.Clone or download
button. Again, make sure you are on your fork. This means that you see [username]/ds-gitflows-[username]
at the top of the page with forked from USGS-R/ds-gitflows-[username]
underneath.cd
) to the location where you would like to create the cloned directory. I would recommend creating a folder somewhere in your D drive to put GitHub projects.git clone [insert URL]
and hit enter, e.g. git clone [email protected]:[username]/ds-gitflows-[username].git
. Note that you cannot CTRL+V to paste into Git Bash. Right click and choose paste instead.You have now successfully cloned your fork! Close this issue and move on to the next one.
In the last issue, we made two commits that added two new sections of the story to our dryville_story.md
file. Huzzah! Everything is peachy. However, while you were doing that, your collaborator also decided to add to the story and they committed before you. Now, we need to once again pull down their changes before moving on.
Action: Pull down your collaborator's most recent additions.
git status
and look for the phrase, "nothing to commit".git pull upstream master
.Your first merge conflict! Everything will be OK, promise :) Close this issue and move on to the next one.
Time to add on to this story. We've done this a couple times now, so the edit-save-add-commit pattern should be getting familiar.
Action: Add text for the next two sections and commit those changes.
git status
in Git Bash right now, you should see a message that says "nothing to commit" and also "Your branch is ahead of 'origin/master' by 1 commit" (that one commit is the one from your collaborator that has not yet been pushed to your fork).dryville_story.md
file on your computer and add the next section of the story, which is called "Your First Flood". Keep the same formatting as before (##
for the title, [inline text](url)
for hyperlinks). Save the file.git status
in Git Bash, you should see the words "modified: dryville_story.md". This means that Git detects your change. You will also see the words "no changes added to commit". This is because we still need to stage our changes. Note that you will still see "Your branch is ahead of 'origin/master' by 1 commit" - we still haven't pushed since we pulled down our collaborator's changes.git add dryville_story.md
to stage these changes so that they can be included in our next commit. Now when you run git status
, you see that the "modified: dryville_story.md" change is listed under "Changes to be committed". We are now ready to make a commit.git commit -m "add your-first-flood section"
to make a commit.git status
again. We should be back to where we started. There is "nothing to commit" because we don't have any additional changes to the repository content - we already committed our changes. However, you will now see that it says "Your branch is ahead of 'origin/master' by 2 commits".##
for the title, [inline text](url)
for hyperlinks). Save the file.git add dryville_story.md
to stage these changes.git commit -m "add storing-water-for-a-rainy-day section"
to make a commit.git status
shows that we are ahead of origin/master
by 3 commits.You have now made two new commits. Close this issue and move on to the next one.
Your PR was merged! Yay! Now, remember that "closing the loop" thing we did after our last PR was merged? Let's do it again.
Merging a PR creates a commit on the canonical repository. Even though the most recent changes were your additions, technically your fork and local repository do not have that "merge" commit and are now out-of-date with the canonical repo. So, we need to close the loop.
Action: Close the loop.
cd
).git pull upstream master
.git status
, we get the message "Your branch is ahead of 'origin/master' by X commits".git push
(or git push origin master
to be explicit).git status
, you should see that everything is up-to-date and there is nothing to commit.You have once again closed the loop after your PR was merged! Close this issue and move to the next one.
With the new changes on your fork, your are now ready to create a pull request (PR). A PR bundles all of your commits together and requests that they be pulled into the canonical repository. When a PR is opened, you typically request that a collaborator review your changes. They can look at your PR and examine how the sum of all of your commits differ from the existing canonical repo. They can make suggestions for revisions to specific lines and ask for changes before they merge your contributions into the main repo.
For now, we will discuss how to open a pull request.
Action: Open a pull request.
https://github.com/[username]/ds-gitflows-[username]
). At the top, you will see a "New pull request button" (see image below). Click that button.base repository
is set to the project canonical repo (USGS-R/ds-gitflows-[username]
in this case) and that the head repository
is set to your fork ([username]/ds-gitflows-[username]
). You also need to verify the commits lists. It should list the two that you just created.#[issue number]
.Reviewers
feature on the right bar and select @lindsayplatt as the reviewer from the drop-down menu. If you do not have the option to add a reviewer, do step 6 and then add the reviewer after.You have now successfully created a PR! Wait for your PR to be reviewed and merged. Once your PR has been merged, close this issue and move on to the next one.
We are now ready to push our local changes (including the resolved merge conflict) up to our fork (aka remote "origin"). Then, we can open a pull request (PR).
Action: Push your changes to your GitHub fork and open a PR.
git status
in Git Bash right now, you should see a message that says "nothing to commit" and also "Your branch is ahead of 'origin/master' by X commits". Our local version has new changes that do not appear in our fork (aka the remote origin
because our local version originated from it).git push origin master
to do so.https://github.com/[username]/ds-gitflows-[username]
) and click on the commits
button (see image below). You should see your new commit messages appear there.base repository
is set to the project canonical repo (USGS-R/ds-gitflows-[username]
) and that the head repository
is set to your fork ([username]/ds-gitflows-[username]
).#[issue number]
.Reviewers
feature on the right bar and select @lindsayplatt as the reviewer from the drop-down menu. If you do not have the option to add a reviewer, do step 9 and then add the reviewer after.You have successfully made a second pull request! Wait for your PR to be reviewed and merged. Once your PR has been merged, close this issue and move on to the next one.
The last topic we are going to cover in this tutorial is the .gitignore
file. These files are used when you have a local file that you don't want to track changes to in a commit or put on GitHub. You simply add the name of the file (including directory structure when applicable) to the .gitignore
file to have Git ignore it (see what they did there?). This practice is often used for intermediate files created by processes within your repo, temporary files, or really large data that need to be stored elsewhere. Additional information about .gitignore
files can be found in this article.
Action: Add a file and then gitignore it.
.gitignore
file that is in your local directory using a text editor (for example, Notepad++). It should be completely empty.wss-wu-dryville.jpg
in your ds-gitflows-[username]
folder.ls
to "list" the items in your current working directory. You should see the following: dryville_story.md
, README.md
and wss-wu-dryville.jpg
. Note that the .gitignore
file doesn't show up with ls
because it is technically a "hidden" file (starts with a .
).git status
. Git shows that wss-wu-dryville.jpg
is an untracked file. We could leave it like that and just try to remember to not commit it, but that seems risky. The more foolproof way is to to add it to the .gitignore
file.wss-wu-dryville.jpg
to the .gitignore
file. Save the file.git status
. It no longer shows wss-wu-dryville.jpg
as an untracked file, but instead shows that we have modified .gitignore
. That is because this file is version controlled - whenever we change it, we push those changes to the master repository so that everyone uses the same one..gitignore
and push to your fork.git add .gitignore
git commit -m "add downloaded image to gitignore"
git push origin master
#[issue number]
. Reminder - you can look back at closed issues to follow along with past instructions by clicking "Issues" and then "Closed" on the GitHub page for the canonical repository.Once you open your PR, wait for it to be reviewed and merged. Once your PR has been merged, close this issue and go to the next one.
Practice makes perfect - let's make a second commit to our local repo.
Action: Add the next section's text and commit this change.
git status
in Git Bash right now, you should see a message that says "nothing to commit" and also "Your branch is ahead of 'origin/master' by 1 commit".dryville_story.md
file on your computer and add the next section of the story, which is called "Dryville's First Water Works". Keep the same formatting as before (##
for the title, [inline text](url)
for hyperlinks). Save the file.git status
in Git Bash, you should see the words "modified: dryville_story.md". This means that Git detects your change. You will also see the words "no changes added to commit". This is because we still need to stage our changes. Note that you will still see "Your branch is ahead of 'origin/master' by 1 commit" - more on that later.git add dryville_story.md
to stage these changes so that they can be included in our next commit. Now when you run git status
, you see that the "modified: dryville_story.md" change is listed under "Changes to be committed". We are now ready to make a commit.git commit -m "add dryvilles-first-water-works section"
to make a second commit.git status
again. We should be back to where we started. There is "nothing to commit" because we don't have any additional changes to the repository content - we already committed our changes. However, you will now see that it says "Your branch is ahead of 'origin/master' by 2 commits". We'll talk about that next.You have now made two commits and recorded your changes with Git! Close this issue and move on to the next one.
Welcome to the end of the hands on tutorial with Git and GitHub! You have now been exposed and practiced the basic workflow that USGS DS uses for collaborating on codebases. See the bottom of this issue for instructions about next steps.
git clone [insert your fork's SSH URL]
.git remote add upstream [insert canonical repo's SSH URL]
.git remote -v
and check that your fork's URL is next to origin
and the canonical repo is listed next to upstream
.git status
git add [insert file name]
. Pro tip: use git add .
to stage all changed files listed with git status
.git commit -m "[commit message here]"
git push origin master
base
repository shows the canonical and the head
repository shows your fork.#[issue number]
in your description.git pull upstream master
<<<<<<< HEAD
and =======
, which is the content that existed before the merge (likely your content).=======
and >>>>>>> [string of letters and numbers]
, which is the content that is trying to be merged with what currently exists locally (likely the remote content).<<<<<<< HEAD
, =======
, and >>>>>>> [string of letters and numbers]
.There is a lot more Git that can be learned, but the above are the basics that will probably be enough for awhile. As you start to get more advanced, there may be some additional concepts/commands you should learn such as,
git stash
, git stash apply
),git checkout -b
), andAction: Once you are done reading this, close this issue and return to the canonical repo's main GitHub page https://github.com/USGS-R/ds-gitflows-[username]
.
We refer to online versions of GitHub repositories as "remotes". If you open Git Bash to your project directory (you may need to cd ds-gitflows-[username]
from the end of the last issue) and run git remote -v
, you will see a list of remotes and their URLs that are currently associated with your local copy. Currently, you have one remote - your fork of the repository - though you will see both a fetch and push option for it. It is referred to as the origin
because your local copy originated from it.
What we need to do now is link the canonical repository to your local copy. This closes the loop and enables you to pull down changes that collaborators have merged to the main repo into your local version. We refer to the online canonical version as the upstream
repo and it is a remote
because it is online.
Action: Link your cloned repository to the upstream remote.
https://github.com/USGS-R/ds-gitflows-[username]
Clone or download
git remote add upstream [insert URL]
, e.g. git remote add upstream https://github.com/USGS-R/ds-gitflows-[username]
. Reminder: you cannot CTRL+V to paste into Git Bash. Right click and choose paste instead.git remote -v
you should see a list with both an upstream
remote and an origin
remote.You have now set up your new project for collaborative development! We are ready to start making changes. Close this issue and move on to the next one.
The master
branch of this repository will soon be renamed from master
to main
, as part of a coordinated change across the USGS-R and USGS-VIZLAB organizations. This is part of a broader effort across the git community to use more inclusive language. For instance, git, GitHub, and GitLab have all changed or are in the process of changing their default branch name.
We will make this change early in the week of February 6, 2022. The purpose of this issue is to give notification of the change and provide information on how to make it go smoothly.
When this change is made, forks and clones of the repository will need to be updated as well. Here are instructions for making the necessary changes.
master
to main
(<your repository> -> Settings -> Branches).First, update your local git configuration so that new repositories created locally will have the correct default branch: git config --global init.defaultBranch main
.
Now, for any forks, do the following:
master
to main
.git branch -m master main
git fetch origin
git branch -u origin/main main
git remote set-head origin -a
The first step in our workflow when working on a new project is to fork the canonical repository. This creates a copy of the repository that is specific to your user on GitHub. Everyone that is working on the project has their own fork of the repository where they can safely make changes without impacting the main code or other contributor's code.
Action: Fork this repo!
https://github.com/USGS-R/ds-gitflows-[username]
.Where should we fork ...
, choose your user account.USGS-R/ds-gitflows-[username]
, you will now see [username]/ds-gitflows-[username]
at the top.Congratulations! You've made your own copy of the main repository. Close this issue and proceed to the next one.
Congratulations - your PR was merged and you have successfully changed the canonical repository. You are almost a Git Pro!
Merging a PR creates a commit on the canonical repository. Even though the most recent changes were your additions, technically your fork and local repository do not have that "merge" commit and are now out-of-date with the canonical repo. So, what we will do now is learn how to close the loop after your PR is merged.
Action: Close the loop.
cd
).git pull upstream master
.git status
, we get the message "Your branch is ahead of 'origin/master' by X commits".git push
(or git push origin master
to be explicit).git status
, you should see that everything is up-to-date and there is nothing to commit.You have successfully closed the loop after your PR was merged! Close this issue and move to the next one.
You are now ready to start contributing your own content to the project! Normally, you would be adding new files, editing lines of code, etc; however, to keep this tutorial programming language-agnostic, we will be editing text in a Markdown document. I think you have already learned about Markdown but if not, visit this quick article to learn about it.
We will be using the dryville_story.md
file to illustrate changes to a repository. First, you will make a change and then save it with Git.
Action: Add text to the story and commit your change.
git status
in Git Bash in your project directory. You should see a message that says "nothing to commit". This means that there are no changes on your local copy and it exactly matches the content on your remote fork (the origin
repo). This is good!dryville_story.md
file on your computer. Any text editor will do, such as Notepad++. Currently, there is a title (denoted by #
) and two sub-headers (denoted by ##
) with text. You can also see the syntax for hyperlinks, [text that appears](link/to/the/website)
. To see how this syntax is rendered on GitHub, open the dryville_story.md
file on GitHub by going to the main repo (https://github.com/USGS-R/ds-gitflows-[username]
) and clicking the file name.##
), the body text, and the appropriate link for the words "over 8 pounds a gallon" to the dryville_story.md
file locally. Save the file.git status
in Git Bash, you should see the words "modified: dryville_story.md". This means that Git detects a new change. At this point, you could run git diff
to visually see the changes you made: red = original, green = changed (if you do this and see a :
at the bottom of your bash window, type q
to get out of the diff view before proceeding). You will also see the words "no changes added to commit". This is because we have not told Git to record these changes; we have not "staged" them.git add dryville_story.md
. Now when you run git status
, you see that the "modified: dryville_story.md" change is listed under "Changes to be committed". We are now ready to make a commit.git commit -m "[insert your message here]"
. For this change, we will run git commit -m "add getting-water-to-your-homes section"
. Any change that was listed under the "Changes to be committed" section when we ran git status
will be included in this commit.git status
again. We should be back to where we started. There is "nothing to commit" because we don't have any additional changes to the repository content - we already committed our only changes. However, you will also see that it says "Your branch is ahead of 'origin/master' by 1 commit". We'll talk about that later.You have now made a commit and recorded your changes with Git! Close this issue and move on to the next one.
Scenario: After your content was merged, a collaborator tells you that they added additional content. You want to keep working, but you don't want to duplicate anything they did. You now need to pull in their content to your local repository before you continue working. Don't worry, we have done this before. We just need to pull down any new changes from the canonical repository!
Action: Pull down a collaborator's contributions to the canonical repository.
https://github.com/USGS-R/ds-gitflows-[username]
and click on "commits"). You should see a new commit that was not created by you.git status
and look for the phrase, "nothing to commit".git pull upstream master
. Remember, the "canonical repository" is referred to as the "upstream" remote in git commands.dryville_story.md
file locally, you should see the "Be Gone, Dirty Water" section is the last in the file.Great! You successfully pulled down contributions that someone else on your team made to the repository. Next, we will add another section. Close this issue and move to the next one.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.