Comments (2)
Build workflow prototype (ssh/scp/7z/powershell)
- Requires 7zip and openssh on windows host. No other requirements.
- Both are available natively and supported by microsoft.
- Files are compressed then transferred using scp over ssh, and decompressed on the windows machine.
- What about using rsync to handle file transfer between linux and windows?
- No native rsync for windows. Rsync over SSH still requires an rsync client available on the target machine.
This is the most efficient and reliable method prototyped so far.
# choco install openssh
# choco install 7zip.commandline
export EC2_INSTANCE="ec2-ip-ad-dr-es"
export EC2_USER="builder"
export PRIVKEY_PATH="./id_rsa.my_private_key"
export LOCAL_DIR="source_dir/"
export TARGET_DIR="builder_dir/"
export EXPORT_NAME="export_archive"
export IMPORT_NAME="import_archive"
export BUILD_COMMAND="ls"
# export
> tar -vczf ${EXPORT_NAME}.tar.gz ${LOCAL_DIR}
> scp -i ${PRIVKEY_PATH} -v \
${EXPORT_NAME}.tar.gz \
${EC2_USER}@${EC2_INSTANCE}.compute-1.compute-1.amazonaws.com:c:/Users/${EC2_USER}/
# unpacking
> ssh -i ${PRIVKEY_PATH} \
${EC2_USER}@${EC2_INSTANCE}.compute-1.amazonaws.com \
7z x .\${EXPORT_NAME}.tar.gz
> ssh -i ${PRIVKEY_PATH} \
${EC2_USER}@${EC2_INSTANCE}.compute-1.amazonaws.com \
7z x .\${EXPORT_NAME}.tar
# run command(s)
> ssh -i ${PRIVKEY_PATH} \
${EC2_USER}@${EC2_INSTANCE}.compute-1.amazonaws.com \
'powershell -NonInteractive -Command cd c:/Users/${EC2_USER}/${EXPORT_NAME}; ${BUILD_COMMAND}'
# pack
> ssh -i ${PRIVKEY_PATH} \
${EC2_USER}@${EC2_INSTANCE}.compute-1.amazonaws.com \
'powershell -NonInteractive -Command 7z a -ttar -so ${IMPORT_NAME}.tar archive_name | 7z a -si ${IMPORT_NAME}.tar.gz'
# import
> scp -i ${PRIVKEY_PATH} -v \
${EC2_USER}@${EC2_INSTANCE}.compute-1.compute-1.amazonaws.com:c:/Users/${EC2_USER}/${IMPORT_NAME}.tar.gz \
${IMPORT_NAME}.tar.gz
> mkdir ${TARGET_DIR}
> tar -vxz --strip-components=1 --file=${IMPORT_NAME}.tar.gz --directory=${IMPORT_NAME}/
# delete artifacts and other cleanup
from hal.
R&D for "powershell remoting over ssh."
1 Install powershell v6 latest on mac:
V6 alpha 14 - https://github.com/PowerShell/PowerShell/releases/tag/v6.0.0-alpha.14 (pkg file)
> powershell
PowerShell
Copyright (C) 2016 Microsoft Corporation. All rights reserved.
PS /Users/skluck/sample-dir> Get-Command New-PSSession -syntax
New-PSSession [[-ComputerName] <string[]>] [-Credential <pscredential>] [-Name <string[]>] [-EnableNetworkAccess] [-ConfigurationName <string>] [-Port <int>] [-UseSSL] [-ApplicationName <string>] [-ThrottleLimit <int>] [-SessionOption <PSSessionOption>] [-Authentication <AuthenticationMechanism>] [-CertificateThumbprint <string>] [<CommonParameters>]
New-PSSession [-VMId] <guid[]> -Credential <pscredential> [-Name <string[]>] [-ConfigurationName <string>] [-ThrottleLimit <int>] [<CommonParameters>]
New-PSSession [-ConnectionUri] <uri[]> [-Credential <pscredential>] [-Name <string[]>] [-EnableNetworkAccess] [-ConfigurationName <string>] [-ThrottleLimit <int>] [-AllowRedirection] [-SessionOption <PSSessionOption>] [-Authentication <AuthenticationMechanism>] [-CertificateThumbprint <string>] [<CommonParameters>]
New-PSSession -Credential <pscredential> -VMName <string[]> [-Name <string[]>] [-ConfigurationName <string>] [-ThrottleLimit <int>] [<CommonParameters>]
New-PSSession [[-Session] <PSSession[]>] [-Name <string[]>] [-EnableNetworkAccess] [-ThrottleLimit <int>] [<CommonParameters>]
New-PSSession -ContainerId <string[]> [-Name <string[]>] [-ConfigurationName <string>] [-RunAsAdministrator] [-ThrottleLimit <int>] [<CommonParameters>]
New-PSSession [-HostName] <string[]> [-Name <string[]>] [-UserName <string>] [-KeyFilePath <string>] [-SSHTransport] [<CommonParameters>]
New-PSSession -SSHConnection <hashtable[]> [-Name <string[]>] [<CommonParameters>]
This option indicates the powershell version does support SSH:
New-PSSession [-HostName] <string[]> [-Name <string[]>] [-UserName <string>] [-KeyFilePath <string>] [-SSHTransport] [<CommonParameters>]
2 Ensure powershell subsystem is installed on windows host sshd_config
:
# override default of no subsystems
Subsystem sftp C:/Program Files/OpenSSH-Win64/sftp-server.exe
Subsystem powershell C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe -sshs -NoLogo -NoProfile
3 Attempt connection from linux (or mac) to windows powershell
PS /Users/skluck/sample-dir> New-PSSession -HostName ec2-ip-ad-dr-es.compute-1.amazonaws.com -UserName builder -KeyFilePath id_rsa.builder
New-PSSession : [ec2-ip-ad-dr-es.compute-1.amazonaws.com] The background process reported an error with the following message: The SSH client session has ended with error message:
packet_write_wait: Connection to ec2-ip-ad-dr-es: Broken pipe.
At line:1 char:1
+ New-PSSession -HostName ec2-ip-ad-dr-es.compute-1.amazonaws.com -Us ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OpenError: (System.Manageme....RemoteRunspace:RemoteRunspace) [New-PSSession], PSRemotingTransportException
+ FullyQualifiedErrorId : 2100,PSSessionOpenFailed
This appears to be a bug due to a recent change between in Win32-openssh
and powershell
.
Related issues here:
from hal.
Related Issues (20)
- Support visualstudio.com / Team Foundation Server
- Support global builds (all environments)
- Remove ability for agent to manually build docker images
- Hal-Core README license badge showing `proprietary` instead of `MIT` HOT 2
- Refactor UI / CSS / Design
- Add support for dotenv / env-based secrets HOT 1
- Add health status endpoint
- Add health status test suite for the API HOT 1
- Create an API prototype in Elixir
- Better API rate limiting
- Hal Core 3.0 missing db constraints HOT 2
- Refactor permission system again
- Update domain model further
- Harden docker security HOT 1
- Document installation and create development infrastructure
- Update hal-agent to use lazy commands
- Consider alternatives to Doctrine for ORM
- Add support for internal users / login with github HOT 1
- Create a real-time event feed using websockets and Poxa HOT 1
- Migration Deletes and Updates HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hal.