rsync filter or exclude to account for .gitignore

[burnett01]

rsync filter or exclude to account for .gitignore

[timocouckuyt]

Hi @Burnett01,

Wauw, thanks for your feedback! The git diff approach seems perfect!

Don't feel obliged, but when you would have some time to write such a script, that would be totally awesome!

Let's keep in touch.

// T

[Burnett01]

Hey, unfortunately I haven't had a chance to get my hands on this. I didn't forget about it so I just wanted to give you an update.
I hope I can find some spare time and simulate your use case.

[Burnett01]

Hey @timocouckuyt thanks for reaching out.

Your approach to getting the .gitignore file recognized by Rsync would be using some sort of conversion. I think you'd write a small shell script that translates the git paths to rsync-filter rules so that

my/file.txt
!other.stuff

would translate to:

+ my/file.txt
- other.stuff
- *

With awk and sed this could be quickly achieved for some basic conversion.

You would then add a new step for that, right before calling the action:

name: Convert .gitignore to filter
run: ./github/converter.sh .gitgnore ./github/myfilter.conf

Script synopsis: converter.sh [inputfile] [outputfile]

Then you could pass the resulting filter to the action:

--delete --filter="merge .github/myfilter.conf"

Let me know if this helps and if you need assistance :)

Greetings Steve

[timocouckuyt]

Oh, great, thanks a lot for the input!

I don't know anything about shell scripting, but I could dive into it, for sure.

Am I correct when I assume that your line in 'run: ' is storing the converted lines into a .conf file? Is there any way of returning the same result inside the --filter option, but without first writing a conf file? So rather inline inside the option value?

Thanks a lot, once again!

// T

[timocouckuyt]

Another take on this: Accounting for .gitignore could be a pain when other companies/freelancers add files and folders to the remote folder over time that should not be deleted and would never be stated in the .gitignore file. In fact, better would be that rsync compares the 2 last commits and performs only these changes.

For example:

• If there's only 1 commit, rsync should create all files and folders appearing in the repo, no deletions.
• 2 files are updated in the repo: rsync only updates these files.
• 1 folder is created, containing 1 file: rsync only creates the folder and adds the file.
• That same folder is removed from the repo, including the file: rsync should only perform a delete of that folder and the file.
• A folder gets renamed: rsync should only account for that renaming.
• etc.

In fact the repo commits should be mirrored into the same actions. But I don't know if that would ever be possible with rsync?

Thanks a lot for your input!

// T

[Burnett01]

Hey @timocouckuyt thanks for your replies.

Yes, basically it should be possible to get Rsync behave the way you described, but not without using some sort of script that prepares everything for --filter. A filter is not required when using the --include and --exclude switches, but then you could hit a limit of max allowed cmd-arg characters. Creating and using an intermediary filter config is thus much neater and reliable. But a filter conf might not be required after all if rsync allows for piping the output of the script to the --filter switch. I gotta do some research on that.

For the script you would take the output of git diff (changed paths only + states like M,D,A) and convert these to rsync filter rules (+/-) and have it create a temporary filter conf (unless the output could be piped to rsync directly).

This issue seems related to #20, so when we have a solution for your issue, the other one will be solved too 😃

If you need help, I can get my hands dirty and write such script in the next week when I'm back at my hometown for a short vacation.

Greetings Steve

[Burnett01]

I haven’t had a chance to look into this as of yet but I‘ll see if I can simulate this scenario in a container.

[timocouckuyt]

Great, @Burnett01 !

[timocouckuyt]

No worries @Burnett01, take your time, I'm glad that you're even considering it.