Giter Site home page Giter Site logo

microsoft / git Goto Github PK

View Code? Open in Web Editor NEW

This project forked from git-for-windows/git

722.0 40.0 83.0 314 MB

A fork of Git containing Microsoft-specific patches.

Home Page: http://git-scm.com/

License: Other

Shell 36.89% Makefile 0.74% C 51.15% C++ 0.51% Perl 5.20% M4 0.17% Emacs Lisp 0.01% Python 0.81% Go 0.07% Tcl 3.78% JavaScript 0.22% CSS 0.05% Roff 0.12% Batchfile 0.05% SmPL 0.06% CMake 0.18%

git's Introduction

microsoft/git and the Scalar CLI

Open in Visual Studio Code Build status

This is microsoft/git, a special Git distribution to support monorepo scenarios. If you are not working in a monorepo, you are likely searching for Git for Windows instead of this codebase.

In addition to the Git command-line interface (CLI), microsoft/git includes the Scalar CLI to further enable working with extremely large repositories. Scalar is a tool to apply the latest recommendations and use the most advanced Git features. You can read the Scalar CLI documentation or read our Scalar user guide including the philosophy of Scalar.

If you encounter problems with microsoft/git, please report them as GitHub issues.

Why is this fork needed?

Git is awesome - it's a fast, scalable, distributed version control system with an unusually rich command set that provides both high-level operations and full access to internals. What more could you ask for?

Well, because Git is a distributed version control system, each Git repository has a copy of all files in the entire history. As large repositories, aka monorepos grow, Git can struggle to manage all that data. As Git commands like status and fetch get slower, developers stop waiting and start switching context. And context switches harm developer productivity.

microsoft/git is focused on addressing these performance woes and making the monorepo developer experience first-class. The Scalar CLI packages all of these recommendations into a simple set of commands.

One major feature that Scalar recommends is partial clone, which reduces the amount of data transferred in order to work with a Git repository. While several services such as GitHub support partial clone, Azure Repos instead has an older version of this functionality called the GVFS protocol. The integration with the GVFS protocol present in microsoft/git is not appropriate to include in the core Git client because partial clone is the official version of that functionality.

Downloading and Installing

If you're working in a monorepo and want to take advantage of the performance boosts in microsoft/git, then you can download the latest version installer for your OS from the Releases page. Alternatively, you can opt to install via the command line, using the below instructions for supported OSes:

Windows

Note: Winget is still in public preview, meaning you currently need to take special installation steps: Either manually install the .appxbundle available at the preview version of App Installer, or participate in the Windows Insider flight ring since winget is available by default on preview versions of Windows.

To install with Winget, run

winget install --id microsoft.git

Double-check that you have the right version by running these commands, which should have the same output:

git version
scalar version

To upgrade microsoft/git, use the following Git command, which will download and install the latest release.

git update-microsoft-git

You may also be alerted with a notification to upgrade, which presents a single-click process for running git update-microsoft-git.

macOS

To install microsoft/git on macOS, first be sure that Homebrew is installed then install the microsoft-git cask with these steps:

brew tap microsoft/git
brew install --cask microsoft-git

Double-check that you have the right version by running these commands, which should have the same output:

git version
scalar version

To upgrade microsoft/git, you can run the necessary brew commands:

brew update
brew upgrade --cask microsoft-git

Or you can run the git update-microsoft-git command, which will run those brew commands for you.

Linux

Ubuntu/Debian distributions

On newer distributions*, you can install using the most recent Debian package. To download and validate the signature of this package, run the following:

# Install needed packages
apt-get install -y curl debsig-verify

# Download public key signature file
curl -s https://api.github.com/repos/microsoft/git/releases/latest \
| grep -E 'browser_download_url.*msft-git-public.asc' \
| cut -d : -f 2,3 \
| tr -d \" \
| xargs -I 'url' curl -L -o msft-git-public.asc 'url'

# De-armor public key signature file
gpg --output msft-git-public.gpg --dearmor msft-git-public.asc

# Note that the fingerprint of this key is "B8F12E25441124E1", which you can
# determine by running:
gpg --show-keys msft-git-public.asc | head -n 2 | tail -n 1 | tail -c 17

# Copy de-armored public key to debsig keyring folder
mkdir /usr/share/debsig/keyrings/B8F12E25441124E1
mv msft-git-public.gpg /usr/share/debsig/keyrings/B8F12E25441124E1/

# Create an appropriate policy file
mkdir /etc/debsig/policies/B8F12E25441124E1
cat > /etc/debsig/policies/B8F12E25441124E1/generic.pol << EOL
<?xml version="1.0"?>
<!DOCTYPE Policy SYSTEM "https://www.debian.org/debsig/1.0/policy.dtd">
<Policy xmlns="https://www.debian.org/debsig/1.0/">
  <Origin Name="Microsoft Git" id="B8F12E25441124E1" Description="Microsoft Git public key"/>
  <Selection>
    <Required Type="origin" File="msft-git-public.gpg" id="B8F12E25441124E1"/>
  </Selection>
  <Verification MinOptional="0">
    <Required Type="origin" File="msft-git-public.gpg" id="B8F12E25441124E1"/>
  </Verification>
</Policy>
EOL

# Download Debian package
curl -s https://api.github.com/repos/microsoft/git/releases/latest \
| grep "browser_download_url.*deb" \
| cut -d : -f 2,3 \
| tr -d \" \
| xargs -I 'url' curl -L -o msft-git.deb 'url'

# Verify
debsig-verify msft-git.deb

# Install
sudo dpkg -i msft-git.deb

Double-check that you have the right version by running these commands, which should have the same output:

git version
scalar version

To upgrade, you will need to repeat these steps to reinstall.

*Older distributions are missing some required dependencies. Even though the package may appear to install successfully, microsoft/ git will not function as expected. If you are running Ubuntu 18.04 or older, please follow the install from source instructions below instead of installing the debian package.

Other distributions

You will need to compile and install microsoft/git from source:

git clone https://github.com/microsoft/git microsoft-git
cd microsoft-git
make -j12 prefix=/usr/local
sudo make -j12 prefix=/usr/local install

For more assistance building Git from source, see the INSTALL file in the core Git project.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

git's People

Contributors

avar avatar benpeart avatar bk2204 avatar chriscool avatar dennisameling avatar derrickstolee avatar dscho avatar felipec avatar gitster avatar hvoigt avatar j6t avatar jeffhostetler avatar jiangxin avatar jnareb avatar jrn avatar kblees avatar mhagger avatar newren avatar patthoyts avatar pclouds avatar peff avatar philipoakley avatar rscharfe avatar spearce avatar srabbelier avatar sschuberth avatar stefanbeller avatar szeder avatar torvalds avatar ttaylorr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

git's Issues

`git merge` reports different result on Mac and Windows

  • I was not able to find an open or closed issue matching what I'm seeing

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
On Mac: git version 2.19.0.gvfs.1.36.gb06df091a3
On Windows: git version 2.19.0.gvfs.1.34.gc7fb556

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

CMD on Windows, Terminal on Mac

<clone the VFS for Git repo>
<Add wilbaker's fork as a remote>
git checkout dont_assert_on_file_attributes_merge_weirdness
git checkout merge_weirdness_master
git checkout dont_assert_on_file_attributes_merge_weirdness
git merge merge_weirdness_master
git status
  • What did you expect to occur after running these commands?

I expected that the output would be the same on both platforms, instead, it was only correct on Windows.

Windows output

C:\Repos\VFSForGit\src>git merge merge_weirdness_master
Removing Scripts/StopDevService.bat
Removing Scripts/StartDevService.bat
Removing Scripts/SetupDevService.bat
Auto-merging ProjFS.Mac/PrjFSKext/PrjFSKext/KauthHandler.cpp
CONFLICT (content): Merge conflict in ProjFS.Mac/PrjFSKext/PrjFSKext/KauthHandler.cpp
Automatic merge failed; fix conflicts and then commit the result.

C:\Repos\VFSForGit\src>git status
On branch dont_assert_on_file_attributes_merge_weirdness
Your branch is up-to-date with 'origin/dont_assert_on_file_attributes_merge_weirdness'.

You have unmerged paths.
  (fix conflicts and run "git commit")
  (use "git merge --abort" to abort the merge)

Changes to be committed:

        modified:   .gitignore
        modified:   GVFS/FastFetch/CheckoutJob.cs
        modified:   GVFS/GVFS.Common/Git/DiffTreeResult.cs
        modified:   GVFS/GVFS.Common/Git/GitObjects.cs
        modified:   GVFS/GVFS.Common/Git/GitProcess.cs
        modified:   GVFS/GVFS.Common/Prefetch/BlobPrefetcher.cs
        modified:   GVFS/GVFS.Common/Prefetch/Git/DiffHelper.cs
        modified:   GVFS/GVFS.FunctionalTests/Tests/EnlistmentPerFixture/PrefetchVerbWithoutSharedCacheTests.cs
        new file:   GVFS/GVFS.Tests/DataSources.cs
        modified:   GVFS/GVFS.UnitTests/Data/forward.txt
        new file:   GVFS/GVFS.UnitTests/Git/GitProcessTests.cs
        modified:   GVFS/GVFS.UnitTests/Prefetch/DiffHelperTests.cs
        modified:   GVFS/GVFS.UnitTests/Prefetch/DiffTreeResultTests.cs
        modified:   GVFS/GVFS.Virtualization/FileSystemCallbacks.cs
        modified:   ProjFS.Mac/PrjFSKext/PrjFSKext/VirtualizationRoots.cpp
        modified:   ProjFS.Mac/PrjFSKext/PrjFSKext/VirtualizationRoots.hpp
        modified:   ProjFS.Mac/PrjFSKext/PrjFSKext/VnodeUtilities.cpp
        modified:   ProjFS.Mac/PrjFSKext/PrjFSKext/VnodeUtilities.hpp
        modified:   Scripts/BuildGVFSForWindows.bat
        new file:   Scripts/InitializeEnvironment.bat
        modified:   Scripts/Mac/BuildGVFSForMac.sh
        modified:   Scripts/Mac/CleanupFunctionalTests.sh
        modified:   Scripts/Mac/DownloadGVFSGit.sh
        modified:   Scripts/Mac/GVFS_Clone.sh
        modified:   Scripts/Mac/GVFS_Mount.sh
        modified:   Scripts/Mac/GVFS_Unmount.sh
        modified:   Scripts/Mac/GetGitVersionNumber.sh
        new file:   Scripts/Mac/InitializeEnvironment.sh
        modified:   Scripts/Mac/NukeBuildOutputs.sh
        modified:   Scripts/Mac/PrepFunctionalTests.sh
        new file:   Scripts/Mac/PublishGit.sh
        modified:   Scripts/Mac/RunFunctionalTests.sh
        modified:   Scripts/NukeBuildOutputs.bat
        modified:   Scripts/ReinstallGVFS.bat
        modified:   Scripts/RestorePackages.bat
        modified:   Scripts/RunFunctionalTests.bat
        modified:   Scripts/RunUnitTests.bat
        deleted:    Scripts/SetupDevService.bat
        deleted:    Scripts/StartDevService.bat
        modified:   Scripts/StopAllServices.bat
        deleted:    Scripts/StopDevService.bat
        modified:   Scripts/UninstallGVFS.bat
        modified:   nuget.config

Unmerged paths:
  (use "git add <file>..." to mark resolution)

        both modified:   ProjFS.Mac/PrjFSKext/PrjFSKext/KauthHandler.cpp


  • What actually happened instead?

Mac Output

~/Repos/VFSForGit/src>git merge merge_weirdness_master
Removing Scripts/StopDevService.bat
Removing Scripts/StartDevService.bat
Removing Scripts/SetupDevService.bat
Auto-merging ProjFS.Mac/PrjFSKext/PrjFSKext/KauthHandler.cpp
CONFLICT (content): Merge conflict in ProjFS.Mac/PrjFSKext/PrjFSKext/KauthHandler.cpp
Automatic merge failed; fix conflicts and then commit the result.
~/Repos/VFSForGit/src>git status
On branch dont_assert_on_file_attributes_merge_weirdness
All conflicts fixed but you are still merging.
  (use "git commit" to conclude merge)

Changes to be committed:

	new file:   GVFS/GVFS.Tests/DataSources.cs
	new file:   GVFS/GVFS.UnitTests/Git/GitProcessTests.cs

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   .gitignore
	modified:   GVFS/FastFetch/CheckoutJob.cs
	modified:   GVFS/GVFS.Common/Git/DiffTreeResult.cs
	modified:   GVFS/GVFS.Common/Git/GitObjects.cs
	modified:   GVFS/GVFS.Common/Git/GitProcess.cs
	modified:   GVFS/GVFS.Common/Prefetch/BlobPrefetcher.cs
	modified:   GVFS/GVFS.Common/Prefetch/Git/DiffHelper.cs
	modified:   GVFS/GVFS.FunctionalTests/Tests/EnlistmentPerFixture/PrefetchVerbWithoutSharedCacheTests.cs
	modified:   GVFS/GVFS.UnitTests/Data/forward.txt
	modified:   GVFS/GVFS.UnitTests/Prefetch/DiffHelperTests.cs
	modified:   GVFS/GVFS.UnitTests/Prefetch/DiffTreeResultTests.cs
	modified:   GVFS/GVFS.Virtualization/FileSystemCallbacks.cs
	modified:   ProjFS.Mac/PrjFSKext/PrjFSKext/KauthHandler.cpp
	modified:   ProjFS.Mac/PrjFSKext/PrjFSKext/VirtualizationRoots.cpp
	modified:   ProjFS.Mac/PrjFSKext/PrjFSKext/VirtualizationRoots.hpp
	modified:   ProjFS.Mac/PrjFSKext/PrjFSKext/VnodeUtilities.cpp
	modified:   ProjFS.Mac/PrjFSKext/PrjFSKext/VnodeUtilities.hpp
	modified:   Scripts/BuildGVFSForWindows.bat
	modified:   Scripts/Mac/BuildGVFSForMac.sh
	modified:   Scripts/Mac/CleanupFunctionalTests.sh
	modified:   Scripts/Mac/DownloadGVFSGit.sh
	modified:   Scripts/Mac/GVFS_Clone.sh
	modified:   Scripts/Mac/GVFS_Mount.sh
	modified:   Scripts/Mac/GVFS_Unmount.sh
	modified:   Scripts/Mac/GetGitVersionNumber.sh
	modified:   Scripts/Mac/NukeBuildOutputs.sh
	modified:   Scripts/Mac/PrepFunctionalTests.sh
	modified:   Scripts/Mac/RunFunctionalTests.sh
	modified:   Scripts/NukeBuildOutputs.bat
	modified:   Scripts/ReinstallGVFS.bat
	modified:   Scripts/RestorePackages.bat
	modified:   Scripts/RunFunctionalTests.bat
	modified:   Scripts/RunUnitTests.bat
	deleted:    Scripts/SetupDevService.bat
	deleted:    Scripts/StartDevService.bat
	modified:   Scripts/StopAllServices.bat
	deleted:    Scripts/StopDevService.bat
	modified:   Scripts/UninstallGVFS.bat
	modified:   nuget.config

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	Scripts/InitializeEnvironment.bat
	Scripts/Mac/InitializeEnvironment.sh
	Scripts/Mac/PublishGit.sh


  • If the problem was occurring with a specific repository, can you provide the
    URL to that repository to help us with testing?

https://github.com/wilbaker/VFSForGit

Add process/file lock to `gvfs-helper prefetch`

As an optimization we could add a file lock to gvfs-helper prefetch to make
concurrent prefetches operate serially.

Ideally, multiple prefetches on a single repo would be hitting the same cache-server
(or origin server) and by default only ask for packs newer than the most recent one
in the shared-cache. So both processes would probably always receive the same
set of pack and waste network bandwidth, since the loser of the race will just discard
their copy of the packs.

A file lock could be added to avoid this.

This is probably only important for the highly-concurrent GVFS test suite.
Normal users are probably unlikely to run concurrent git fetch commands.

Usefulness of multi-pack-index outside of VFS for Git

The multi-pack-index feature is really useful when we are in an environment with many pack-files. This is the case with VFS for Git, due to the prefetch model. Most Git environments can simply repack into one pack-file, so do not need the feature.

Question: Are there other environments where many pack-files occur and could use a multi-pack-index?

Please leave comments below if you believe the multi-pack-index can help your situation!

Mac: tab completion doesn't work for branches

I have a mac now! Trying to use Git on Terminal has some limitations. Specifically, I'm used to using tab-completion for branch names.

$ git push fork myB<tab>
$ git push fork myBranch

This does not appear to work on the Mac in Terminal. Is it something wrong with our install, or is the supplied tab-completion not wired correctly for Mac?

Scalar clone: make 404s on `gvfs/config` silent

I ran scalar clone against a GitHub URL and saw these lines in the output:

error: config: (http:404) Not Found
error: config: (http:404) Not Found

These are likely related to the /gvfs/config endpoint, which doesn't exist on GitHub (and many others). This error is likely being output from Curl, but we should pass the correct option to make this be silent.

git status --deserialize can report stale results if excludes file is modified

When changes are made to the global excludes file (defaults to "$XDG_HOME/git/excludes") or
the repo-local excludes file (".git/info/excludes"), an existing status serialization cache should be
invalidated and subsequent "git status --deserialize" should reject it and fallback to a normal
(scanning) status run.

Changes in either of these files can change the result of "git status" when there are untracked
or ignored files present in the workdir. The status serialization cache file contains results for
untracked and ignored files relative to the exclude settings when "git status --serialize" was
executed
. When either of these files change, the cached results should be discarded.

Add metadata for user-global and repo-local excludes-files to the status serialization header
(like we do for the .git/index mtime). Use this to invalidate the status cache if either excludes-file
changes and cause "git status --deserialize" to reject the cache and fallback to a normal status run.

The GVFS service daemone watches for changes to files within the projected workdir and will
invalidate the status cache appropriately. This includes watching for changes to the various
per-directory .gitignore files.

The issue here is necessary because the 2 excludes file are outside the workdir and primarily
for the global excludes file since the pathname can come from the "core.excludesFile" config
setting.

Limitations: "git status --deserialize" will not invalidate the status cache if any of the
per-directory .gitignore files are changed or if there are any other changes within the
workdir. Therefore, it may report stale results. "git status --deserialize" would need
something similar to the full untracked-cache index extension (that records the directory
mtimes) to detect that. We don't use the untracked-cache here because one of the goals
of the status serialization feature was to avoid reading the index (in addition to unnecessary
disk scanning). This is only possible because the GVFS service daemon handles these cases
for us.

git update-microsoft-git does not install updates

  • [ X ] I was not able to find an open or closed issue matching what I'm seeing

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
$ git --version --build-options

git version 2.33.0.vfs.0.0
cpu: x86_64
built from commit: 24f9d5c95d3ab8fe65553b8d488e1b287eda8e42
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
$ cmd.exe /c ver

Microsoft Windows [Version 10.0.19044.1526]
  • What options did you set as part of the installation? Or did you choose the
    defaults?
# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

Output is from -> type "C:\Program Files\Git\etc\install-options.txt"
Editor Option: VIM
Custom Editor Path:
Default Branch Option:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Core
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Disabled
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

Not that I'm aware.

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

PowerShell 7.2.1

git --version
git update-microsoft-git
git --version

//Output:
PS C:\Users\doug.murphy> git --version
git version 2.33.0.vfs.0.0
PS C:\Users\doug.murphy> git update-microsoft-git
Git 2.33.0.vfs.0.0 (64-bit)
PS C:\Users\doug.murphy> git --version
git version 2.33.0.vfs.0.0

  • What did you expect to occur after running these commands?

That my installation of Git would be updated to the latest from this GitHub repo (v2.35.1.vfs.0.0 at the time of this writing)

  • What actually happened instead?

Nothing. The update did not occur. No mention of any available update.

  • If the problem was occurring with a specific repository, can you provide the
    URL to that repository to help us with testing?

N/A

Pre/post-command hooks and post-index-change hook might inherit stdin

If a user pastes multiple commands into a terminal, they expect those commands to be run in order.

However, the later lines become part of consumed stdin, depending on the existence of a post-command hook or a post-index-change hook. This might be due to those hook implementations taking stdin in a way they shouldn't, but maybe Git shouldn't be passing stdin to these hooks in the first place.

read-tree: add progress indicators

git read-tree -mu HEAD is the mechanism used for updating a working directory after updating a sparse-checkout file. However, there is no progress indicator for it doing the work. This can be a lot of work if the working directory grows a lot and many files need to be written.

Add a progress indicator to git read-tree. Don't forget:

  1. to include a --[no-]progress option to the command line.

  2. to turn progress off by default (to match existing behavior).

After #180 is complete, update the builtin code for git sparse-checkout to pass --progress (unless the command is run without a terminal, i.e.isatty(2) == 0).

`git difftool -d` with Beyond Compare 4: cannot spawn .git/hooks/virtual-filesystem: No such file or directory

  • I was not able to find an open or closed issue matching what I'm seeing

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

64-bit

$ git --version --build-options

git version 2.17.1.gvfs.2.1.6.gdc051a1
cpu: x86_64
built from commit: dc051a1f5beb8ce0b22051afd9be86a0541a534a
sizeof-long: 4
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

64-bit Windows 10

  • What options did you set as part of the installation? Or did you choose the
    defaults?
# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

M:\>type "C:\Program Files\Git\etc\install-options.txt"
Editor Option: VIM
Path Option: Cmd
SSH Option: OpenSSH
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Disabled
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

This is with a VFSForGit repo.

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

CMD

<modify some files>
git difftool -d
  • What did you expect to occur after running these commands?

Beyond Compare 4 would launch, and allow me to modify and save source files.

  • What actually happened instead?

Beyond Compare 4 launched, but changes made in the right pane were lost (and the error message below was displayed).

M:\gvfsDiffTest2\src>git difftool -d
error: cannot spawn .git/hooks/virtual-filesystem: No such file or directory
fatal: unable to load virtual file system
fatal: diff-files did not exit properly
  • If the problem was occurring with a specific repository, can you provide the
    URL to that repository to help us with testing?

This is not repo specific, but is specific to using VFSForGit.

core.autocrlf = false not working

Setup

  • Which version of microsoft/git are you using? Is it 32-bit or 64-bit?
$ git --version --build-options

git version 2.36.1.vfs.0.0
cpu: x86_64
built from commit: c6886cdb2dd3bc5a365e74d2c5fa15fb93ff8342
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

Are you using Scalar or VFS for Git?

If VFS for Git, then what version?

$ gvfs version

zsh: command not found: gvfs
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
Mac OS 12.3.1
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

** insert your response here **

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

zsh

git status   
git diff --staged
  • What did you expect to occur after running these commands?

I tried copying some files from other git repositories to a new Git repository using VFS, after copying, "git status" always showed them changed, but "git diff -staged" was always empty, after researching I found out that this was the cause of crlf.

However, when I tried to disable crlf checks on the new system, it didn't help.

  • What actually happened instead?

crlf is always treated as modified, I tried adding safecrlf = true but it didn't work either.

  • If the problem was occurring with a specific repository, can you specify
    the repository?

Improve documentation: Is this fork still needed for non-GVFS workflows?

This is a suggestion to update the description with information on whether (and why) this fork is still needed if you have an up-to-date upstream Git client.

From my understanding, the original project was GVFS, which is practically dead. The next thing after that was Scalar, which has so far been superseded by this repository. Now unless I am using a remote via the GVFS protocol (which I'm not), it is not quite clear to me whether I still need this fork. The scalar CLI command seems to be a useful yet optional thin-wrapper around the initial git clone that sets up recommended .git/config settings, but other than that, I don't really see any difference between this fork and the upstream project, both in terms of functionality and performance. It would be great if the documentation could be more clear on this.

Xcode's Source Control issue

Hi, I am working on the Office monorepo and we found that Xcode's Source Control(libgit2 1.3) doesn't work with microsoft/git.

But after a lot of testing I accidentally found that Xcode works as long as I use the official git(2.37.0) to run status command once, can you take a look at this and fix it? Thanks

Bring Back DeleteMe repo

  • I was not able to find an open or closed issue matching what I'm seeing

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
$ git --version --build-options

** insert your machine's response here **
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
$ cmd.exe /c ver

** insert your machine's response here **
  • What options did you set as part of the installation? Or did you choose the
    defaults?
# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

** insert your machine's response here **
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

** insert your response here **

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

** insert your response here **

** insert your commands here **
  • What did you expect to occur after running these commands?

** insert here **

  • What actually happened instead?

** insert here **

  • If the problem was occurring with a specific repository, can you provide the
    URL to that repository to help us with testing?

** insert URL here **

"vfs" in git version string breaks cmake git version parser

  • [ X ] I was not able to find an open or closed issue matching what I'm seeing

Setup

64-bit git on Ubuntu 18.04 64 bit

$ git --version --build-options

git version 2.29.0.vfs.0.0
cpu: x86_64
built from commit: 3c7b47771c6afc8e8093f066fe29ff80fcf0b8ac
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh

  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
$ cmd.exe /c ver

Ubuntu Linux - using this git build with Scalar
  • What options did you set as part of the installation? Or did you choose the
    defaults?
# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

N/A
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

bash

git clone 
https://github.com/leggedrobotics/SimBenchmark
cd SimBenchmark/lib
mkdir build
cd build
cmake ..

  • What did you expect to occur after running these commands?

cmake run successfully completes

  • What actually happened instead?

CMake Error at /usr/share/cmake-3.19/Modules/ExternalProject.cmake:2449 (message):
error: git version 1.6.5 or later required for --recursive flag with 'git
submodule ...': GIT_VERSION_STRING=''

  • If the problem was occurring with a specific repository, can you provide the
    URL to that repository to help us with testing?

https://github.com/leggedrobotics/SimBenchmark

Commit-graph: use two-dimensional graph positions

As we build incremental commit-graph files, the idea of "graph position" is incredibly tricky to manage. It's not difficult to work with a single chain of incremental commit-graph files, but in a scenario with many alternates (for example, a fork network on a Git server) it can be beneficial to have multiple "commit-graph chains". In these chains, there can be multiple commits with the "same" graph position depending on which chain we started from.

We can use a two-dimensional graph position to work with this: (chain index, position in chain).

This will likely require moving graph_pos into a 64-bit commit slab.

GVFS-Helper: Fail harder on failed write?

A build machine recently had failures in git status with the following logs:

Running git status...
open2: exec of watchman -j --no-pretty failed at /Volumes/DataB/og1/src/.git/hooks/query-watchman line 47.
error: gvfs-helper error: 'could not create tempfile: '/Users/bud/.scalarCache/id_ba47400f-eebe-4850-a8bf-ef694ef81414/pack/tempPacks/t-20200625-025012-165469-0000-5.temp': from GET 0751ff206365fa35c07e154bfe4500e2db5f5676'
error: gvfs-helper died of signal 15
error: gvfs-helper error: 'could not create tempfile: '/Users/bud/.scalarCache/id_ba47400f-eebe-4850-a8bf-ef694ef81414/pack/tempPacks/t-20200625-025012-631659-0000-5.temp': from POST'
error: gvfs-helper died of signal 15

However, the git status appears to have completed "successfully", which was confusing. Perhaps we should fail harder when the git-gvfs-helper has this kind of failure.

v2.37.x Windows installers do not appear to be codesigned correctly

I don't get any codesigning/SmartScreen warnings when downloading+installing Git-2.36.1.vfs.0.0-64-bit.exe, but I do when downloading either Git-2.37.0.vfs.0.0-64-bit.exe or Git-2.37.1.vfs.0.0-64-bit.exe. On the surface, the two EXEs appear to have digital signatures on them, but when running them, the elevation dialog says Unknown Publisher, and Edge/SmartScreen give big warnings about it being an unknown publisher. Release notes say these releases had build/release workflow changes, so I wanted to bubble this up.

image

image

image

Untracked symlinks not reported by 'git status' when virtualization hook in use

  • I was not able to find an open or closed issue matching what I'm seeing

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
$ git --version --build-options

git version 2.18.0.gvfs.1.26.gf61ade4ce8
cpu: x86_64
built from commit: 36c283e844cdcc1293983df635d0164cde010c2a
sizeof-long: 8

  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
$ cmd.exe /c ver

Windows 10 insider release and Mac 10.13.6

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

CMD on Windows and Terminal on Mac.

1. Clone repo with VFS for Git
2. Create a new symbolic link
3. Run git status
  • What did you expect to occur after running these commands?

Git status should report the new link as untracked.

  • What actually happened instead?

Status did not report the new link.

Git: multi-pack-index builtin always shows progress

Splitting this out as a separate issue from #177

The multi-pack-index builtin shows progress during its output. There is no way to turn it off right now. This means that when we have an error, stderr is filled with progress indicators and the actual error message may be truncated in our logging.

Update git to allow --no-progress and use this argument in the background operations.

Post-command hooks are sometimes being called twice

We've had reports of the post-command hook being called twice.

The current theory is:

[...] something called die/exit while we were running
child[3]. There's a race in run_post_command_hook() that would allow child[4]
to still be started.

Sparse-checkout builtin is not atomic

The git sparse-checkout set subcommand updates the sparse-checkout then tries a git read-tree -mu HEAD. If there are conflicts in the files being added or removed, then the read-tree will fail. However, the sparse-checkout file is still updated, which will lead to a surprise when eventually the checkout succeeds.

Make the set subcommand more atomic by backing up the sparse-checkout file and reverting the changes on a failed update.

TODO Code scan git-gvfs-helper for file handle leak

While doing a very large Scalar clone and checkout on the Office repo I ran into an out-of-file handles error on the Mac. gvfs-helper was attempting to download 445 blob packfiles and
died after 245.

...
Receiving packfile 243/445 with 4000 objects (bytes received): 181524673, done.
Receiving packfile 244/445 with 4000 objects (bytes received): 192938717, done.
Receiving packfile 245/445 with 4000 objects (bytes received): 185614872, done.
fatal: open /dev/null failed: Too many open files
error: packet write with format failed

git-gvfs-helper: be more careful around bad prefetch packs

A user reported an error of the form

error: index file C:/.scalarCache/id_ba47400f-eebe-4850-a8bf-ef694ef81414/pack/prefetch-1869771333-6573743e645f5f32352e4d6f76654e6578742829.idx is too small

The .idx file had 0 bytes, so my default thought was an fsync() issue.

I recommended that they delete the .idx and the multi-pack-index then run git index-pack on that pack file. However, that failed, as well:

C:\...\src>git index-pack C:/.scalarCache/id_ba47400f-eebe-4850-a8bf-ef694ef81414/pack/prefetch-1869771333-6573743e645f5f32352e4d6f76654e6578742829.pack
fatal: pack signature mismatch

This is possibly due to not catching an fsync inside git-gvfs-helper when the download cut short. Possibly the index-pack command we run failed, but for some reason allowed the prefetch to complete with success.

We need to make this more robust!

sparse-checkout: correct behavior on dirty status

When running git sparse-checkout set with a dirty status, we have the following behavior:

  1. Upstream: do not run. Clean status is required.
  2. Here: Run, but possibly overwrite staged changes.

We need to allow the command to run with a dirty status, but to do the following:

  • Do not run if there is a dirty status outside of the intersection of the old cone and the new cone. (i.e. if something would change skip-worktree bits but has dirty status, THEN complain)
  • If there is a dirty status inside the intersection of the two cones, then preserve the index state.

git commands are slow with many calls to git_config_get_virtualfilesystem => GetEnvironmentVariableW

  • I was not able to find an open or closed issue matching what I'm seeing

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
X:\Office\src>git --version --build-options
git version 2.33.0.vfs.0.0
cpu: x86_64
built from commit: 24f9d5c95d3ab8fe65553b8d488e1b287eda8e42
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
X:\Office\src>cmd.exe /c ver
Microsoft Windows [Version 10.0.19043.1288]
  • What options did you set as part of the installation? Or did you choose the
    defaults?
X:\Office\src>type "C:\Program Files\Git\etc\install-options.txt"
Editor Option: VIM
Custom Editor Path:
Default Branch Option:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Core
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Disabled
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

Scalar / sparse-checkout is being used.

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

cmd

X:\Office\src>git config --unset core.virtualfilesystem
X:\Office\src>timethis git status
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:39:27 2021
On branch namespace/mybranch Your branch is up to date with 'origin/namespace/mybranch'.
You are in a sparse checkout with 8% of tracked files present.
nothing to commit, working tree clean
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:39:27 2021
TimeThis :      End Time :  Fri Oct 15 14:39:40 2021

TimeThis :  Elapsed Time :  00:00:13.498


X:\Office\src>git config --add core.virtualfilesystem ""
X:\Office\src>timethis git status
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:39:43 2021
On branch namespace/mybranch Your branch is up to date with 'origin/namespace/mybranch'.
You are in a sparse checkout with 8% of tracked files present.
nothing to commit, working tree clean
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:39:43 2021
TimeThis :      End Time :  Fri Oct 15 14:39:46 2021

TimeThis :  Elapsed Time :  00:00:02.831


X:\Office\src>git config --unset core.virtualfilesystem
X:\Office\src>timethis git status
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:39:59 2021
On branch namespace/mybranch Your branch is up to date with 'origin/namespace/mybranch'.
You are in a sparse checkout with 8% of tracked files present.
nothing to commit, working tree clean
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:39:59 2021
TimeThis :      End Time :  Fri Oct 15 14:40:13 2021

TimeThis :  Elapsed Time :  00:00:13.408


X:\Office\src>git config --add core.virtualfilesystem ""
X:\Office\src>timethis git status
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:40:22 2021
On branch namespace/mybranch Your branch is up to date with 'origin/namespace/mybranch'.
You are in a sparse checkout with 8% of tracked files present.
nothing to commit, working tree clean
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:40:22 2021
TimeThis :      End Time :  Fri Oct 15 14:40:25 2021

TimeThis :  Elapsed Time :  00:00:02.780


X:\Office\src>git config --unset core.virtualfilesystem
X:\Office\src>timethis git status
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:41:06 2021
On branch namespace/mybranch Your branch is up to date with 'origin/namespace/mybranch'.
You are in a sparse checkout with 8% of tracked files present.
nothing to commit, working tree clean
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:41:06 2021
TimeThis :      End Time :  Fri Oct 15 14:41:19 2021

TimeThis :  Elapsed Time :  00:00:13.370


X:\Office\src>git config --add core.virtualfilesystem ""
X:\Office\src>timethis git status
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:41:26 2021
On branch namespace/mybranch Your branch is up to date with 'origin/namespace/mybranch'.
You are in a sparse checkout with 8% of tracked files present.
nothing to commit, working tree clean
TimeThis :  Command Line :  git status 
TimeThis :    Start Time :  Fri Oct 15 14:41:26 2021
TimeThis :      End Time :  Fri Oct 15 14:41:28 2021

TimeThis :  Elapsed Time :  00:00:02.759
  • What did you expect to occur after running these commands?

The git status performance to be ~2-3 seconds

  • What actually happened instead?

The git status performance is around ~13 seconds

  • If the problem was occurring with a specific repository, can you provide the
    URL to that repository to help us with testing?

Office: Bug

Odd compile error with RES_STARTEND and regex.h version on MSVC build

I think I've found a problem in the Makefile for v2.27.0. (I haven't tracked it back to when it first appeared yet.)

Basically, when building the MSVC version (using make MSVC=1 DEBUG=1) I'm seeing an error

...
fuzz-commit-graph.c
C:\work\msft\git-compat-util.h(1135): fatal error C1189:
    #error:  "Git requires REG_STARTEND support. Compile with NO_REGEX=NeedsStartEnd"
...

However, we're already defining that symbol in config.mak.uname in the MSVC section.

Hacking the Makefile to include -E and looking at the CPP output shows that the compiler is
finding the wrong version of regex.h. It is finding:

...
__pragma(pack(pop))
#line 189 "C:\\work\\msft\\git-compat-util.h"
#line 1 "C:\\git-sdk-64\\mingw64\\include\\regex.h"
...

and NOT the one in ./compat/regex/regex.h

Note that the MSVC section in config.mak.uname is adding -Icompat -Icompat/regex to COMPAT_FLAGS at https://github.com/microsoft/git/blob/vfs-2.27.0/config.mak.uname#L451 (and later again at https://github.com/microsoft/git/blob/vfs-2.27.0/Makefile#L1793 )

Later, COMPAT_CFLAGS gets appended to BASIC_CFLAGS at https://github.com/microsoft/git/blob/vfs-2.27.0/Makefile#L2004

BUT At https://github.com/microsoft/git/blob/vfs-2.27.0/Makefile#L1357-L1381
(and when CURLDIR is mingw64) we append -I/mingw64/include to BASIC_CFLAGS,
but this happens before COMPAT_CFLAGS is appended.

Therefore, the compiler sees ... -I/mingw64/include -Icompat -Icompat/regex ...
and tries to use the wrong version of regex.h which leads to the compiler error.

I'm not sure if this problem is unique to a conflict with CURL. It looks like it Expat, ZLib,
IConv, and a few others could similarly update BASIC_CFLAGS before the values from
COMPAT_CFLAGS are brought in.

Thoughts??

`git log -L` is very slow

A user reported git log -L N,M:file taking a long time (~7 minutes after cleaning up the object directory). The same git log file command was almost immediate.

Check if there are easy performance wins to reduce the computation time when using -L.

git status should not fail when it is unable to open cache file

In microsoft/VFSForGit#663, a functional test failed to run status because git encountered an error (access denied) when reading from a status cache file.

Reading status from a cache file is a best effort operation. If the file does not exist, we fall back to computing a fresh status. The intention is that failing to read from the cache file is also a best effort, and we should not terminate if we are unable to read an existing file.

We should be more resilient to these types of errors, and fallback to computing full status gracefully rather than terminating. The issue here is the utility xopen function used by status cache deserialization logic will terminate if it encounters an issue reading the file.

GVFS Prefetch runs into rate limits on Azure DevOps

  • I was not able to find an open or closed issue matching what I'm seeing

image

The global consumption limit is 200 TSTUs within a sliding five-minute window. Exceeding this gets us throttled down to 0-12 TSTUs every 5 minutes. Can we change the prefetch to stay just under the threshold?

https://docs.microsoft.com/en-us/azure/devops/integrate/concepts/rate-limits?view=azure-devops

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
    64 bit
$ git --version --build-options

git version 2.33.0.vfs.0.3
cpu: x86_64
built from commit: 7e354fb19cf049c46b94750aa639efd1468b94a1
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
    Windows 11 64 bit
$ cmd.exe /c ver

Microsoft Windows [Version 10.0.22000.434]
  • What options did you set as part of the installation? Or did you choose the
    defaults?
    Defaults
# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"

Editor Option: VisualStudioCode
Custom Editor Path:
Default Branch Option:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Core
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Disabled
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

Microsoft internal

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Powershell

scalar clone --full-clone (some large azure devops repo link)
  • What did you expect to occur after running these commands?

The repo should have been downloaded within ~20 mins

  • What actually happened instead?

It ran into rate limits and got throttled by a lot.

  • If the problem was occurring with a specific repository, can you provide the
    URL to that repository to help us with testing?

https://o365exchange.visualstudio.com/O365%20Core/_git/Nibiru

Sparse-checkout: respect `core.ignoreCase`

For this issue, assume we are in "cone mode" with core.sparseCheckoutCone enabled.

The git sparse-checkout set command currently converts a list of paths into prefix patterns that will update the working directory to only include files in those folders (or adjacent to those folders). If the given patterns do not match in case, then those folders will not be included.

To resolve these case issues, perform the following check in the set subcommand when core.ignoreCase is enabled:

  1. For each pattern, check if it matches an existing pattern in the index.
  2. If there is no matching pattern, attempt finding an alternate case.
  3. If an alternate casing is found, then save that pattern instead. (This prevents performance issues in the many more workdir-changing commands such as git checkout).
  4. (Optional) If no alternate casing is found, then fail, perhaps only when --strict is specified.

Git: commit-graph builtin always shows progress

The commit-graph builtin show progress during its output. There is no way to turn it off right now. This means that when we have an error, stderr is filled with progress indicators and the actual error message may be truncated in our logging.

Update git to allow --no-progress and use this argument in the background operations.

Changed-path Bloom filter follow-ups

Here are some things to investigate:

  • Write changed-path Bloom filters with config. See this WIP.
  • Update test coverage:
    • Write --changed-paths in pack order.
    • Make config for 512-change limit. Test going over the limit.
  • Consider batch size?
  • What happens if we have changed-path filters and then fetch.writeCommitGraph triggers?
  • Document the file format very carefully.

Investigate fetch and gvfs-helper

In #240
an issue was uncovered where a git fetch would try to use gvfs-helper prefetch (to the cache-server), receive 0 new objects, and then try a remote-https / fetch-pack directly to the origin server. This would fail because a gvfs-aware origin server does not allow normal fetches and requires use of the gvfs-protocol. This server error prevented fetch from attempting a gvfs-style missing object (either individual or batch) fetch. The behavior of gvfs-helper seems to be slightly different from the original read-object-hook in this regards.

The fix in #240 does address the problem.

The purpose of this issue is to investigate performance. This has only been observed when the origin server is ahead of the cache server. So the ref exchange may have very new objects that are not yet present on the cache server. It seems like we might:
[1] either have gvfs-helper do the fallback to main when it receives a 0 object prefetch from the cache-server. this might prevent the foreground fetch from needing to run the fetch-pack at all.
[2] have fetch do the second prefetch to the main server (rather than hiding it inside gvfs-helper).
[3] other??

My concern here is that the individual missing object fetches following the (now no-op) fetch-pack might be more expensive than a gvfs-prefetch, but I haven't investigated the actual behavior yet.

"Permission denied" while trying to install on Windows

Operating System: Windows

OS Version: Microsoft Windows [Version 10.0.19043.1288]

Description

Describe the bug: what happened? why would you consider it broken?

Ran the Windows git installer with git 2.32.0.vfs.0.2 already installed. During the install, hit [...]:

Unable to set system config "init.defaultBranch":="master": exit code 4
stderr:
error: could not write config file C:/Program Files/Git/etc/gitconfig: Permission denied

Unclear why there would be a permissions error here -- the config file does exist and is not read-only. Like the rest of the program files directory, only admins can write to it but the installer was elevated.

Originally posted by @marcelais in #441 (comment)

Git reset HEAD~1 does not leave modified files in working tree

  • Git reset HEAD~1 in newer git versions does not leave modified files in working tree, files doesn't show up until additional local modifications are made

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
git version 2.35.1.vfs.0.0

Edit: This user is also using VFS for Git.

  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
Windows 10 and 11 64bit
  • What options did you set as part of the installation? Or did you choose the
    defaults?
default

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

both CMD and Bash

2.33.0 2.35.1
D:\os.2020.3\src>git status
warning: unable to find all commit-graph files
On branch official/main
Your branch is up to date with 'origin/official/main'.

nothing to commit, working tree clean

D:\os.2020.3\src>git reset HEAD~1
warning: unable to find all commit-graph files

D:\os.2020.3\src>git status
Refresh index: 100% (3130996/3130996), done.
warning: unable to find all commit-graph files
On branch official/main
Your branch and 'origin/official/main' refer to different commits.
(use "git status --ahead-behind" for details)

Changes not staged for commit:
(use "git add ..." to update what will be committed)
(use "git restore ..." to discard changes in working directory)

modified: sdpublic/misc/Labels/main_label.xml

no changes added to commit (use "git add" and/or "git commit -a")
D:\os.2020.4\src>git reset HEAD~1
warning: unable to find all commit-graph files

D:\os.2020.4\src>git status
warning: unable to find all commit-graph files
On branch official/main
Your branch and 'origin/official/main' refer to different commits.
(use "git status --ahead-behind" for details)

nothing to commit, working tree clean

Unable to get scalar clone to use test cache-server

Setup

  • Which version of microsoft/git are you using? Is it 32-bit or 64-bit?
$ git --version --build-options

git version 2.36.1.vfs.0.0
cpu: x86_64
built from commit: c6886cdb2dd3bc5a365e74d2c5fa15fb93ff8342
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

Are you using Scalar or VFS for Git?

Yes, Scalar.

$ scalar version
git version 2.36.1.vfs.0.0

If VFS for Git, then what version?

$ gvfs version
GVFS 1.0.21014.1
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

Microsoft Windows [Version 10.0.22000.739]

  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

GVFS Proxy setup as a VM. Windows Server 2019. ADO Server. Response from cache server url/gvfs/info

{
  "version": "18.181.32404.7",
  "serverState": "Ready",
  "maintenanceServiceStatus": "Running",
  "currentRequests": 1,
  "repos": [
    {
      "repoId": "49b0c9f4-555f-4624-8157-a57e6df513b3",
      "proxyUser": "ProxyService\\4030e0bc-a40a-4855-9915-712eb44836d6",
      "state": "None",
      "fetchFrequency": "00:05:00",
      "fetchProgress": []
    }
  ],
  "totalPackResponseHits": 2,
  "totalPackResponseMisses": 0,
  "totalLooseObjectMemoryCacheHits": 0,
  "totalLooseObjectMemoryCacheMisses": 7,
  "totalLooseObjectMemoryCacheBytes": 1579329,
  "totalObjectSizeMemoryCacheHits": 0,
  "totalObjectSizeMemoryCacheMisses": 0,
  "totalObjectSizeMemoryCacheBytes": 0,
  "totalLooseObjectDiskHits": 0,
  "totalLooseObjectDiskMisses": 7,
  "totalLooseObjectRemoteMisses": 0,
  "totalLooseObjectDiskCacheBytes": 14413824,
  "currentPrefetchRequests": 0,
  "currentSingleObjectRequests": 0,
  "currentMultiObjectRequests": 0,
  "currentObjectSizesRequests": 0
}

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Git Bash, or CMD.

scalar clone --local-cache-path=/d/other_repos/lrt/scalar_cache/1 --cache-server-url=https://test-1js-proxy/49b0c9f4-555f-4624-8157-a57e6df513b3 https://dev.azure.com/office/office/_git/1JS
  • What did you expect to occur after running these commands?

Expected the cache server to be used, to make the perf reasonable.

  • What actually happened instead?

Instead its falling back on the origin url. There are some really long delays as well between outputting the cache-server url and starting the prefetch ( which also takes a long time, but thats another issue ).

I noticed the ssl cert error in the trace2 log (attached). The cert for the cache server is registered on the machine I am trying to clone from as a Trusted Root Cert. I can browse the server with Edge and hit the gvfs/info point without getting any cert errors. Name resolution only works because I added it to my hosts.ini file locally.

You can also see the long delays in the perf trace attached as well.

Full command response:

$ time scalar clone --local-cache-path=/d/other_repos/lrt/scalar_cache/1 --cache-server-url=https://test-1js-proxy/49b0c9f4-555f-4624-8157-a57e6df513b3 https://dev.azure.com/office/office/_git/1JS
Initialized empty Git repository in D:/other_repos/lrt/sc_proxy/1JS/src/.git/
Receiving vsts/info (bytes received):   0% (0/89555), done.
Receiving vsts/info (bytes received): 100% (824/824), done.
Cache server URL: https://test-1js-proxy/49b0c9f4-555f-4624-8157-a57e6df513b3
Prefetch 0 (1970-01-01 00:00:00 +0000) (bytes received): 9486622286, done.
Indexing objects: 100% (2572567/2572567), done.
Resolving deltas: 100% (1543167/1543167), done.
warning: fetch normally indicates which branches had a forced update,
but that check has been disabled; to re-enable, use '--show-forced-updates'
flag or run 'git config fetch.showForcedUpdates true'
Receiving packfile 1/1 with 14 objects (bytes received): 177073, done.
Indexing objects: 100% (14/14), done.
Switched to a new branch 'main'
branch 'main' set up to track 'origin/main'.
Your branch is up to date with 'origin/main'.

real    18m31.981s
user    0m0.000s
sys     0m0.046s
  • If the problem was occurring with a specific repository, can you specify
    the repository?

    • Public repo: insert URL here
    • Windows monorepo
    • Office monorepo
    • Other Microsoft-internal repo: 1JS
    • Other internal repo.

git_perf.log

multi-pack-index expire doesn't do anything if --objects-dir contains backward slashes

VFSForGit runs PackfileMaintenanceStep periodically, but content of pack subdirectory in .gvfsCache remains unchanged.
The reason is that git multi-pack-index expire command doesn't work with backward slash paths for --object-dir (see reproduction example below).
So VFSForGit runs expire with backward slashes and it doesn't do anything.
It seems git behavior should not depend on paths format.

Setup

  • Which version of microsoft/git are you using? Is it 32-bit or 64-bit?
git version 2.36.0.vfs.0.0
cpu: x86_64
built from commit: 5323cc6ccf2923f4f41da713ea9d1125704dec5b
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

Are you using Scalar or VFS for Git?

VFS for Git

If VFS for Git, then what version?

$ gvfs version

v1.0.21229.1
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
$ cmd.exe /c ver

Microsoft Windows [Version 10.0.19044.1645]
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

no

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

CMD

If you run multi-pack-index expire with backward slashes for --object-dir command do nothing

c:\Windows>"C:\Program Files\Git\mingw64\bin\git.exe" --git-dir="F:\Projects\Monorepo\src\.git" -c core.virtualizeobjects=false multi-pack-index expire --object-dir="F:\.gvfsCache\d7745fd3f2f047bab6827f5b183fdbf0\gitObjects"
13:16:41.308775 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
13:16:41.310776 run-command.c:1549      run_processes_parallel: preparing to run up to 1 tasks
13:16:41.310776 run-command.c:654       trace: run_command: F:/Projects/Monorepo/src/.git/hooks/pre-command.exe multi-pack-index expire '--object-dir=F:\.gvfsCache\d7745fd3f2f047bab6827f5b183fdbf0\gitObjects' --git-pid=31564
13:16:41.404016 run-command.c:1581      run_processes_parallel: done
13:16:41.404016 git.c:531               trace: built-in: git multi-pack-index expire '--object-dir=F:\.gvfsCache\d7745fd3f2f047bab6827f5b183fdbf0\gitObjects'
13:16:41.410589 run-command.c:1549      run_processes_parallel: preparing to run up to 1 tasks
13:16:41.410589 run-command.c:654       trace: run_command: F:/Projects/Monorepo/src/.git/hooks/post-command.exe multi-pack-index expire '--object-dir=F:\.gvfsCache\d7745fd3f2f047bab6827f5b183fdbf0\gitObjects' --git-pid=31564 --exit_code=0
13:16:41.497558 run-command.c:1581      run_processes_parallel: done

Command starts doing smth with forward slashes for --object-dir:

c:\Windows>"C:\Program Files\Git\mingw64\bin\git.exe" --git-dir="F:\Projects\Monorepo\src\.git" -c core.virtualizeobjects=false multi-pack-index expire --object-dir="F:/.gvfsCache/d7745fd3f2f047bab6827f5b183fdbf0/gitObjects"
13:16:56.612496 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
13:16:56.615528 run-command.c:1549      run_processes_parallel: preparing to run up to 1 tasks
13:16:56.615528 run-command.c:654       trace: run_command: F:/Projects/Monorepo/src/.git/hooks/pre-command.exe multi-pack-index expire --object-dir=F:/.gvfsCache/d7745fd3f2f047bab6827f5b183fdbf0/gitObjects --git-pid=34208
13:16:56.715399 run-command.c:1581      run_processes_parallel: done
13:16:56.715399 git.c:531               trace: built-in: git multi-pack-index expire --object-dir=F:/.gvfsCache/d7745fd3f2f047bab6827f5b183fdbf0/gitObjects
Unlink of file 'F:/.gvfsCache/d7745fd3f2f047bab6827f5b183fdbf0/gitObjects/pack/from-loose-922b0b1f773d613ed0ae0a493433d10461ae7c4f.idx' failed. Should I try again? (y/n)
  • If the problem was occurring with a specific repository, can you specify
    the repository?

    • Public repo: insert URL here
    • Windows monorepo
    • Office monorepo
    • Other Microsoft-internal repo: insert name here
    • Other internal repo.

warning: fetch normally indicates which branches had a forced update

Setup

$ git --version --build-options

git version 2.37.1.vfs.0.1
cpu: x86_64
built from commit: 196b322ff5c5f86d9c1e16b91dda9518ac7a7afe
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

Are you using Scalar or VFS for Git?

Uuuuh... neither, I think? I'm using this fork of Git with a normal Git repo on BitBucket ๐Ÿ™‚

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.19042.1889]
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

N/A

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

CMD

git pull
  • What did you expect to occur after running these commands?

No warnings

  • What actually happened instead?

I get the following warning:

C:\sources\>git pull
warning: fetch normally indicates which branches had a forced update,
but that check has been disabled; to re-enable, use '--show-forced-updates'
flag or run 'git config fetch.showForcedUpdates true'
Already up to date.

I believe this warnings started occurring after I switched from Git for Windows to Microsoft's fork of Git, but I'm not 100% sure. If this warning is due to switching to Microsoft's fork of Git, then I think that running scalar initialize should automatically set fetch.showForcedUpdates to true or false ๐Ÿ™‚

Access violation in git.exe when running status

  • I was not able to find an open or closed issue matching what I'm seeing

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
git version 2.22.0.vfs.1.1.57.gbaf16c8

Details on the access violation from @jeffhostetler:

looks like a null repository pointer inside a wt_status structure. and it's only triggered if there are unmerged files and a rebase is in progress when we try to stat() the ".git/MERGE_MSG" file

git prevents pulling upstream due to non-existent local changes

  • I was not able to find an open or closed issue matching what I'm seeing

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

64-bit. However, this also repros on the macOS build of Git.

$ git --version --build-options
git version 2.34.0.vfs.0.0
cpu: x86_64
built from commit: 599a56ae91cee374b95d19c73aeead6ea18ea7a2
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
$ cmd.exe /c ver
Microsoft Windows [Version 10.0.19044.1348]
  • What options did you set as part of the installation? Or did you choose the
    defaults?
# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
Editor Option: VIM
Custom Editor Path:
Default Branch Option:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Core
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Disabled
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

I already had a sparse-checkout repository before using this version of Git.

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Bash

git pull origin <my-branch>

Unfortunately, this issue happened after using this version of git for a while and continued until I added the affected file paths to my sparse checkout list.

  • What did you expect to occur after running these commands?

Git pulls from the upstream

  • What actually happened instead?

Git didn't pull because files were changed that didn't exist locally. These file paths were not in my sparse checkout list. For instance, devapk/excel/en-us existed in the sparse-checkout list, but not devapk/excel/ja-jp.pseudo

git pull origin <my-branch>
Prefetch 1638298835 (2021-11-30 19:00:35 +0000) (bytes received): 8, done.
From https://dev.azure.com/office/office/_git/Office
 * branch                      official/appleauto -> FETCH_HEAD
warning: Fetch normally indicates which branches had a forced update,
but that check has been disabled. To re-enable, use '--show-forced-updates'
flag or run 'git config fetch.showForcedUpdates true'.
error: Your local changes to the following files would be overwritten by merge:
        devapk/excel/ja-jp.pseudo/product/build.gradle
        devapk/excel/ja-jp.pseudo/product/sources
        devapk/officemobile/ja-jp.pseudo/product/build.gradle
        devapk/officemobile/ja-jp.pseudo/product/sources
        devapk/onenote/ja-jp.pseudo/product/modernonenote/apk/build.gradle
        devapk/onenote/ja-jp.pseudo/product/modernonenote/apk/sources
        devapk/ppt/ja-jp.pseudo/product/build.gradle
        devapk/ppt/ja-jp.pseudo/product/sources
        devapk/rio/ar-sa.pseudo/build.gradle
        devapk/word/ja-jp.pseudo/product/build.gradle
        devapk/word/ja-jp.pseudo/product/sources
Please commit your changes or stash them before you merge.
Aborting
Updating 608889a9daba..bac8a2bed9c1
Calling 'git fetch' to pick up upstream refs...
Prefetch 1638298835 (2021-11-30 19:00:35 +0000) (bytes received): 8, done.
warning: Fetch normally indicates which branches had a forced update,
but that check has been disabled. To re-enable, use '--show-forced-updates'
flag or run 'git config fetch.showForcedUpdates true'.
  • If the problem was occurring with a specific repository, can you provide the
    URL to that repository to help us with testing?

https://office.visualstudio.com/Office/_git/Office

Sparse-checkout: recover gracefully from canceled 'set' command

A user reported a situation like the following:

$ git sparse-checkout set <dir1> <dir2> ... <dirN>
Receiving packfile 1/33 with 4000 objects (bytes received): 1068014587, done
^C

$ git sparse-checkout set
Receiving packfile 1/32 with 4000 objects (bytes received): 876319629, done.

So, somehow the first command stored some state of the sparse-checkout file but did not fully succeed in downloading the blob data, and that is affecting the second command.

What confuses me is that we already have logic in the sparse-checkout builtin that should not be committing a pattern set to disk until after the working directory is updated to match. Perhaps there is something wrong with that logic.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.