skatsuta/athenai

Have fun with Amazon Athena from command line! 🕊

athenai's Issues

Manage third party libraries using package management tool

https://github.com/golang/go/wiki/PackageManagementTools

• Godeps
• Glide

Number each Query and show the number as prefix of logging

2017/06/22 00:00:00.000 file.go [Query #1 uuid] ...

Abstract filter

Add support for an option to output results in CSV format

athenai run --format csv "QUERY..."

Add more tests for readline using readlineCloser interface

Call defer userCancelCtx in RunQuery

Create, list, run, delete named queries

Design

• athenai named create
• athenai named list
• athenai named run
• athenai named delete

Use peco to select.

Support Windows

• Test on Windows using Appveyor
• Release Windows binary after confirming it works on Windows

Release automation using goreleaser

goreleaser/goreleaser: Deliver Go binaries as fast and easily as possible
https://github.com/goreleaser/goreleaser

Add `version` command or `--version` flag to show version

Limit concurrent query execution requests up to 5 by default

Service Limits

By default, concurrency limits on your account allow you to run five concurrent queries at a time.

Add a flag to configure this: --parallel or --concurrent

Update help description

• Update short and long description.
• Add usage examples.

Add GoReport badge to README

After open source.

Add safeBuffer to protect String() method from data race

type safeBuffer struct {
  mu sync.Mutex
  bytes.Buffer
}

func (b *safeBuffer) String() string {
  b.mu.Lock()
  defer b.mu.Unlock()
  return b.Buffer.String()
}

Setup coverage report service

Add a badge to README.

Show QueryExecutionId in a result

Cancel executions if Ctrl-C is pressed

References:

• A go example to stop a worker goroutine when Ctrl-C is pressed - Gist
• Make Ctrl+C cancel the context.Context – Mat Ryer – Medium

Add support for encrypting query results on S3

Design

$ athenai run --encrypt SSE_KMS --kms <KMS_KEY_ARN> ...

• --encrypt/-e: specify the type of encryption for query results.
  • Valid values: SSE_S3, SSE_KMS or CSE_KMS
• --kms/-k: KMS key ARN or ID. Only used for SSE_KMS and CSE_KMS.

Reference: EncryptionConfiguration - Amazon Athena API Reference

Implementation

1. Add usage of encryption into README
2. Add flags --encrypt and --kms
3. Add flag values to StartQueryExecution API parameters
4. Add tests
5. Confirm query results are encrypted on S3
6. Update --help usage description on run subcommand

Support config file

• region
• database
• output / location
• format / output
• profile
• section (ini)

The location is $HOME/.athenai/config.

Print version in debug mode

Create animated demo gifs

Use Recordit: Record screencasts fast & free! with GIF Support!.

Fix a bug where TestShowResults sometime panics by index out of range

TestShowResults sometimes panics by index out of range.
Investigate and fix the cause.

--- FAIL: TestShowResults (0.00s)
panic: runtime error: index out of range [recovered]
	panic: runtime error: index out of range

goroutine 118 [running]:
testing.tRunner.func1(0xc42029fad0)
	/usr/local/go/src/testing/testing.go:622 +0x55f
panic(0xadd680, 0xea0b00)
	/usr/local/go/src/runtime/panic.go:489 +0x2f0
github.com/skatsuta/athenai/core.(*stubFilter).Run(0xc42025d800, 0xe76f60, 0xc420086d68, 0xb622be, 0x1)
	/go/src/github.com/skatsuta/athenai/core/athenai_test.go:547 +0x330
github.com/skatsuta/athenai/core.(*Athenai).filterQueryExecutions(0xc4201ca4d0, 0xc42025fa00, 0x0, 0x32, 0x0, 0x32, 0x0, 0x0, 0x0)
	github.com/skatsuta/athenai/core/_test/_obj_test/athenai.go:517 +0x95e
github.com/skatsuta/athenai/core.(*Athenai).selectQueryExecutions(0xc4201ca4d0, 0xe76f20, 0xc4201b5cc0, 0x0, 0x0, 0x0, 0x0, 0x0)
	github.com/skatsuta/athenai/core/_test/_obj_test/athenai.go:568 +0x289
github.com/skatsuta/athenai/core.(*Athenai).ShowResults(0xc4201ca4d0)
	github.com/skatsuta/athenai/core/_test/_obj_test/athenai.go:613 +0x227
github.com/skatsuta/athenai/core.TestShowResults(0xc42029fad0)
	/go/src/github.com/skatsuta/athenai/core/athenai_test.go:751 +0x2928
testing.tRunner(0xc42029fad0, 0xb81100)
	/usr/local/go/src/testing/testing.go:657 +0x108
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:697 +0x544
FAIL	github.com/skatsuta/athenai/core	0.170s

Return an array of arrays instead of a channel from Result#Rows

Create all-in-one array and append it to the table using AppendBulk.

Use bufio.Writer to buffer I/O

Support input from stdin

Read from stdin only if no argument is given.

$ athenai run < file.sql

Protect out writer from data race

See log package

Set log level in command func intead of in newClient()

Add support for showing raw files downloaded from S3

Create Homebrew formula repo

• See Homebrew documentation on tap
• Create skatsuta/homebrew-athenai repo and place Formula in it.

Create another goroutine which prints running messages

And add --silent flag to disable this

Add cache for query results and execution info

Use spinner instead of progress dots while waiting for query executions

https://github.com/briandowns/spinner

Set shorter WaitInterval in athenai_test.go to shorten test time

Add support for reading SQL statements from an SQL file

athenai run file://sample.sql

Show real-time query statuses

Queued:

[QUEUED] QUERY;

Running:

[RUNNING] QUERY;

Succeeded:

[SUCCEEDED] QUERY;

Show query execution results

• Use peco to select
• athenai show

Add `--order` flag to arrange query results in the order of given SQL statements

See 7ee9eac

Add an option to run query in background

athenai run --no-wait "QUERY..."

Avoid using package scoped variables

• Peter Bourgon · A theory of modern Go
• Go, without package scoped variables | Dave Cheney

Print table header

Scan ColumnInfo at first and then skip the first row if it's same as the header.

Show query execution id for each statement after StartQueryExecution API succeeds

Run multiple SQL statements concurrently

Split multiple statements by ;, run them concurrently and show the results in the original order.

Do not print results if a query is DML

Check whether the length of ColumnInfo is 0.

Add interactive mode in run command

$ athenai run
athenai> 

Support saving the output to a file

athenai run --output file

Use sync.Once when calling Athenai#setupREPL

http://marcio.io/2015/07/singleton-pattern-in-go/

Fix description of each subcommand

Create logo

Update readme

• Add contributing process
• Add development process
• Add changelog
• Update installation steps
• Update required IAM permissions
  • http://docs.aws.amazon.com/athena/latest/ug/access.html#amazonathenafullaccess-managed-policy-contents

Add support for tab completion