iawia002/lux

👾 Fast and simple video download library and CLI tool written in Go

lux's Issues

Some video in bilibili.com can not download.

platform: windows 10 1709 x64
version: 0.4.1
url: https://www.bilibili.com/video/av1892780
error message:

λ annie.exe -d https://www.bilibili.com/video/av1892780

URL:         https://www.bilibili.com/video/av1892780
Method:      GET
Headers:     http.Header{
    "Accept-Encoding": {"gzip,deflate,sdch"},
    "Accept-Language": {"en-US,en;q=0.8"},
    "User-Agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"},
    "Referer":         {"https://www.bilibili.com/video/av1892780"},
    "Accept":          {"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
    "Accept-Charset":  {"UTF-8,*;q=0.5"},
}
Status Code: 200

URL:         https://interface.bilibili.com/v2/playurl?appkey=84956560bc028eb7&cid=3522533&otype=json&qn=116&quality=116&type=&sign=9e72c1d0f3bc5f8183a1b22f13879118
Method:      GET
Headers:     http.Header{
    "Accept-Encoding": {"gzip,deflate,sdch"},
    "Accept-Language": {"en-US,en;q=0.8"},
    "User-Agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"},
    "Referer":         {"https://interface.bilibili.com/v2/playurl?appkey=84956560bc028eb7&cid=3522533&otype=json&qn=116&quality=116&type=&sign=9e72c1d0f3bc5f8183a1b22f13879118"},
    "Accept":          {"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
    "Accept-Charset":  {"UTF-8,*;q=0.5"},
}
Status Code: 200

   Site:   哔哩哔哩 bilibili.com
  Title:   【醋醋×祈Inory】星期五的早安\(^o^) ~【生日作】 高清
   Type:   video
Quality:   高清 1080P
   Size:   50.63 MiB (53091354 Bytes)

 0 B / 50.63 MiB [--------------------------------------------------------------------------------------------]   0.00% URL:         http://cn-bj2-cc-v-09.acgvideo.com/vg1/upgcxcode/33/25/3522533/3522533-1-80.flv?expires=1521550800&platform=pc&ssig=L4vqFYouWcDAotmrx7I12g&oi=3742674228&nfa=ZGlYLwTu0dW3o1gJGPmYTQ==&dynamic=1&hfa=2021902754&hfb=Yjk5ZmZjM2M1YzY4ZjAwYTMzMTIzYmIyNWY4ODJkNWI=&trid=4287ead4f1eb49d6ba51f77c54be948f
Method:      GET
Headers:     http.Header{
    "User-Agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"},
    "Accept":          {"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
    "Accept-Charset":  {"UTF-8,*;q=0.5"},
    "Accept-Encoding": {"gzip,deflate,sdch"},
    "Accept-Language": {"en-US,en;q=0.8"},
    "Referer":         {"https://www.bilibili.com/video/av1892780"},
}
Status Code: 200
 0 B / 50.63 MiB [--------------------------------------------------------------------------------------------]   0.00%2018/03/20 19:03:13 Error while downloading: http://cn-bj2-cc-v-09.acgvideo.com/vg1/upgcxcode/33/25/3522533/3522533-1-80.flv?expires=1521550800&platform=pc&ssig=L4vqFYouWcDAotmrx7I12g&oi=3742674228&nfa=ZGlYLwTu0dW3o1gJGPmYTQ==&dynamic=1&hfa=2021902754&hfb=Yjk5ZmZjM2M1YzY4ZjAwYTMzMTIzYmIyNWY4ODJkNWI=&trid=4287ead4f1eb49d6ba51f77c54be948f, invalid argument

Thanks a lot~

Facebook Video Downloader Option

Option to allow download video from Facebook.

programming errors need panic

不知道这些算不算bug

cannot rename the downloaded file

annie -x http://127.0.0.1:1080 https://www.youtube.com/watch?v=rVEMTxg_LrU

Site: YouTube youtube.com
Title: 一生所愛《西遊記之仙履奇緣》主题曲 片尾曲 MV HD 720p
Type: video
Quality: hd720
Size: 81.63 MiB (85595140 Bytes)

81.63 MiB / 81.63 MiB [===========================================================================] 100.00% 8.64 MiB/s2018/03/19 21:28:03 rename 一生所愛《西遊記之仙履奇緣》主题曲 片尾曲 MV HD 720p.mp4.download 一生所愛《西遊記之仙履奇緣》主题曲 片尾曲 MV HD 720p.mp4: The process cannot access the file because it is being used by another process.

maybe the encoding issue in the file name.

[youku] the video URLs in API data is not complete

It seems Youku returned only a few minutes of video data.

some broken URLs:

• http://v.youku.com/v_show/id_XMzQ2NjUyNjYwNA==.html

ccode related? https://github.com/iawia002/annie/blob/master/extractors/youku.go#L43

Package for Void linux

Hi !

For Void linux users a package is now available, see commit. 🎉

xbps-install -S annie

Regards.

能增加个显示下载链接的功能不

rt

http://ip:port urls problem?

Running the following seem to result in an error

./annie -d http://3.18.168.212:7734/file.mp3

annie: version 0.6.3, A fast, simple and clean video downloader.

panic: runtime error: index out of range

goroutine 1 [running]:
github.com/iawia002/annie/utils.Domain(0x7ffefaa7744d, 0x13, 0xc420142180, 0x0)
	/XXXXXXX/go/src/github.com/iawia002/annie/utils/utils.go:58 +0xa3
main.main()
	/XXXXXXX/go/src/github.com/iawia002/annie/main.go:54 +0x1ca

GOOS=linux / GOARCH=amd64 / CGO_ENABLED=0 (if thats of any help)

Thanks

[feature request] any plan for FreeBSD support?

Thank you for this gorgeous work, I love it.

It seems FreeBSD is not supported yet, and the darwin binary shows 'Exec format error' on FreeBSD--it works on macOS only, just as expected.

I tried annie linux-amd64 binary(v0.4.1) on FreeBSD 11.1-RELEASE-p8 with CentOS 7 compatibility layer, only to get the 'Segmentation fault' error.

I know this project is on its early stage and FreeBSD is much less popular than Linux, but if you'd like to take native FreeBSD support into consideration, I'll greatly appreciate that.

FreeBSD用的人少，但它确实是一款优秀的操作系统，发这个issue也算是给它打个广告吧 :)

Create a telegram group for developers, to communicate and for general questions.

GOOD JOB!

支持列表还是有点少啊

flag provided but not defined: -f

Hey, when I try to download with

$ annie -f default https://www.youtube.com/watch?v=x2P7nDtXg-A

I got message like this : flag provided but not defined: -f
And if

$ annie -h

The message show without -f flag

Usage of annie:
  -c string
    	Cookie
  -d	Debug mode
  -i	Information only
  -p	Download playlist
  -r string
    	Use specified Referrer
  -s string
    	SOCKS5 proxy
  -v	Show version
  -x string
    	HTTP proxy

annie: version 0.5.1
MacOS High Sierra 10.13.3

Add input option for overwriting the file with the same name or not

files with the same file name but different file size

Panic downloading video

With 4315740 I got the following output downloading a YouTube playlist, behind an HTTP proxy :

panic: Get https://www.youtube.com/watch?v=lLOvB1qFE1E&gl=US&hl=en&has_verified=1&bpctr=9999999999: Bad Gateway

goroutine 1 [running]:
github.com/iawia002/annie/request.Request(0x70a4cf, 0x3, 0xc4204a0240, 0x57, 0x0, 0x0, 0x0, 0xc4204a0240)
	/home/lamg/go/src/github.com/iawia002/annie/request/request.go:88 +0xc7b
github.com/iawia002/annie/request.Get(0xc4204a0240, 0x57, 0x0, 0x0)
	/home/lamg/go/src/github.com/iawia002/annie/request/request.go:111 +0x7b
github.com/iawia002/annie/extractors.youtubeDownload(0xc4204a01e0, 0x53, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/lamg/go/src/github.com/iawia002/annie/extractors/youtube.go:71 +0x16b
github.com/iawia002/annie/extractors.Youtube(0x7ffcacfc62d0, 0x44)
	/home/lamg/go/src/github.com/iawia002/annie/extractors/youtube.go:52 +0x21c
main.main()
	/home/lamg/go/src/github.com/iawia002/annie/main.go:60 +0x428

New Design

Hi I have made a icon to help you in your project and to better present the program you have developed. If you want any change, you can contact me without hesitation. I hope you use it.

use pretty print for debug info

https://github.com/kr/pretty

[iqiyi] adapt vip cookies

add an option for print extracted data

need pretty print #9

弄了个很繁琐的半次元批量下载处理的网页

用于Windows系统 生成批处理的文件有助于下载归档

@echo off
if not exist "115_2147393" goto 0
exit
:0
md "115_2147393"
cd 115_2147393
annie https://bcy.net/coser/detail/115/2147393
cd..
if not exist "9378_2156617" goto 1
exit
:1
md "9378_2156617"
cd 9378_2156617
annie https://bcy.net/coser/detail/9378/2156617
cd..

说明文档

优酷下载失败

annie https://v.youku.com/v_show/id_XMzQ2NzU2NTg0MA.html -d
panic: runtime error: index out of range

goroutine 1 [running]:
github.com/iawia002/annie/extractors.genData(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x6, 0x8, ...)
/Users/mac/iawia002/go/src/github.com/iawia002/annie/extractors/youku.go:83 +0x4e7
github.com/iawia002/annie/extractors.Youku(0x7fff88f748d3, 0x32, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/mac/iawia002/go/src/github.com/iawia002/annie/extractors/youku.go:111 +0x185
main.main()
/Users/mac/iawia002/go/src/github.com/iawia002/annie/main.go:55 +0x247

大神什么能不能加一个重命名文件功能？

RT

优酷下一半出错

···root@vagrant-ubuntu-trusty-64:/data/python# ./annie_linux_amd64 http://v.youku.com/v_show/id_XMTg2MDkwMjU2OA==.html

Site: 优酷 youku.com
Title: 驴得水
Type: video
Quality: mp4hd2 1280x544
Size: 1011.24 MiB (1060363118 Bytes)

23.68 MiB / 1011.24 MiB [=>-----------------------------------------------------] 2.34% 250.36 KiB/s 1h7m18s 24.51 MiB / 1011.24 MiB [=>------------------------------------------------------] 2.42% 250.94 KiB/s 1h7m5s2018/03/23 16:08:14 Error while downloading: http://vali.cp31.ott.cibntv.net/67712A7A56F4271A343555873/0300012300584E4FFB7BBE05CF07DD5ED554FA-344E-44D9-060F-3977925FDBEC.flv?ccode=0503&duration=203&expire=18000&psid=e8c7535193b2acd84aed20dc2bb6f577&ups_client_netip=3b6c4885&ups_ts=1521793175&ups_userid=&utid=lqY7EyAiCSUCAcu7o+J2eEgs&vid=XMTg2MDkwMjU2OA%3D%3D&vkey=A8dd72981db8c7c95bbb552f14947736f&s=92d6ad9a806711e6b9bb, net/http: request canceled (Client.Timeout exceeded while reading body)
···

建议使用gox发布arm相关版本

不要小看ARM党啦~
修改gox代码，把arm放开即可。
还有aarch64么么哒~

请问是怎么解析出视频的，什么原理？

youtube-dl

There is already youtube-dl project, which despite of the name supports multiple sites, not just YouTube.

Annie's README should provide comparison with youtube-dl or at least mention it as alternative.

Select and download available YouTube video formats

A feature similar to youtube-dl -F which gives a list of available formats, and then allows downloading a specific one using youtube-dl -f number_that_identifies_format

Download YouTube playlists

Feature request.

error is So dry

https://github.com/iawia002/annie/blob/master/main.go#L34

	if len(args) < 1 {
		fmt.Println("error") // so dry
		return
	}

may be you could Add same like annie -h etc, not just error

Rethink the name and naming conventions for data structures in extractors

• Simple and meaningful
• Camel-Case
• Use all upper case or all lower case to represent abbreviated words(URL or url, not Url)

Audio only from youtube.

Hello,

Is there any we can only download audio from Youtube in mp3 format. ?

Thanks

arm64 version

Hi, i would like to propose your software as a Qnap NAS qpkg

is it possible that you propose a precompiled arm64 version ?

does the actual arm version is working on armv5, or only armv7 ?

download video from bilibili.com which url not end with av+id will not exit after downloading.

as the title says, tests below:

tony@tony-VirtualBox:~/下载$ 
   Site:   哔哩哔哩 bilibili.com
  Title:   【Perfume】 Music Station （2018.02.23） 无限未来
   Type:   video
Quality:   高清 720P
   Size:   52.13 MiB (54661154 Bytes)

 52.13 MiB / 52.13 MiB [============================] 100.00% 540.23 KiB/s 1m38s

[1]+  已完成               annie https://www.bilibili.com/video/av19976437?from=search

after downloading, the process will not exit, you should enter the "enter" key to exit.

Youku download error

` Site: 优酷 youku.com
Title: 圆桌派 第三季：第十一集 烂片：怎样算烂？为何会烂？
Type: video
Quality: mp4hd2 1280x720
Size: 322.98 MiB (338664257 Bytes)

39.60 MiB / 322.98 MiB [===>--------------------------] 12.26% 27.53 MiB/s 1s
Merging video parts into 圆桌派 第三季：第十一集 烂片：怎样算烂？为何会烂？.mp4`

我看你用的方法和 http://www.maoyun.tv/mdparse/index.php?id=http://v.youku.com/v_show/id_XMzQ2NjUyNjYwNA==.html 是一样的，没区别，但是他能获取到正确的地址并且播放。

我建议可以从这种解析接口里获取下载地址。

视频合并问题

刚开始没装FFmpeg,下了个电影,无法合并,就又去下载安装了FFmpeg,那时宿舍刚好断网,没网程序报错,不能合并已下载的视频

bilibili: the main download function need return the data as list

[downloader] Separate the Download function, pass VideoData through channels

请问cookies.txt如何获取，具体如何使用~~

请问cookies.txt如何获取，具体如何使用~~

我从IE导出的cookies.txt不能用，出错~~

feature request: download video from multiple url

Feature request:
i have video.txt file which have several video link, so is this possible to download all of video in particular format?
something like..

cat video.txt
video_link:format_which_i_want_to_download
http://videosite.comm/?watch?=klfjsldkf:1080p
http://videosite.comm/?watch?=ddsfsdf:420p
http://videosite.comm/?watch?=aedsfc:720p

bilibili: request canceled

annie https://www.bilibili.com/video/av17748345

Site: 哔哩哔哩 bilibili.com
Title: 【Aimer】Aimer Live in BUDOKAN "blanc et noir" 【中日双语字幕】
Type: video
Quality: 高清 1080P
Size: 1807.59 MiB (1895391009 Bytes)

829.44 MiB / 1.77 GiB [====================================>-------------------------------------------] 45.89% 8.29 MiB/s 1m56s2018/03/21 18:23:10 Error while downloading: http://hk-hkbn7.acgvideo.com/vg7/6/de/28975875-5-80.flv?expires=1521634800&platform=pc&ssig=UqtWX_k7xdW6BizHsphstQ&oi=2946464944&nfa=ZGlYLwTu0dW3o1gJGPmYTQ==&dynamic=1&hfa=2021724512&hfb=Yjk5ZmZjM2M1YzY4ZjAwYTMzMTIzYmIyNWY4ODJkNWI=&trid=806ca226c0bf4373a9ace3a79c1f11ed, net/http: request canceled (Client.Timeout exceeded while reading body)

尝试下载两次但都是在同一个地方失败。好像是链接超时, 但不知道具体原因。（啊再一次沉迷于go的并发机制，决心好好学一下。

Archlinux Support

Hello,

I just wrote a package build for Archlinux(https://aur.archlinux.org/packages/annie/), maybe can be useful for archusers if you mention that in README/install

Regards,
Joaquin

爱奇艺视频下载bug

http://m.iqiyi.com/v_19rrnnoyps.html#vfrm=30-28-15-7
以前用you-get也不行，总是下载一段就错误中断

extract only necessary fomats

refer #83

Only one format:

• bcy
• bilibili
• douyin
• douyu
• pixivision
• tumblr

Multiple formats:

• youtube
• facebook
• iqiyi
• mgtv
• vimeo
• youku

下载出现错误

Type: video
Quality: 高清 1080P
Size: 177.54 MiB (186162839 Bytes)

159.20 MiB / 177.54 MiB [======================================================================================>---------] 89.67% 1.59 MiB/s 10s
2018/03/18 12:40:23 Error while downloading: http://cn-jsnt-dx-v-03.acgvideo.com/vg0/c/61/34219071-2-80.flv?expires=1521354900&platform=pc&ssig=wFbUO0UouG_OrdSSKw_oew&oi=3070288114&nfa=ZGlYLwTu0dW3o1gJGPmYTQ==&dynamic=1&hfa=2023672908&hfb=Yjk5ZmZjM2M1YzY4ZjAwYTMzMTIzYmIyNWY4ODJkNWI=, net/http: request canceled (Client.Timeout exceeded while reading body)**

browser extension

just one click, construct the parameters(with cookies), then open the terminal

Write documentation tailored for newcomers allowing them to contribute to the project

brew support

can I install annie using Homebrew in the future?
thanks for your library.

issue when parse some ytb url

annie -d -x http://127.0.0.1:1080 https://www.youtube.com/watch?v=TUk7q9zRoJI

URL:         https://www.youtube.com/watch?v=TUk7q9zRoJI&gl=US&hl=en&has_verified=1&bpctr=9999999999
Method:      GET
Headers:     http.Header{
    "Accept-Charset":  {"UTF-8,*;q=0.5"},
    "Accept-Encoding": {"gzip,deflate,sdch"},
    "Accept-Language": {"en-US,en;q=0.8"},
    "User-Agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"},
    "Accept":          {"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
    "Referer":         {"https://www.youtube.com/watch?v=TUk7q9zRoJI&gl=US&hl=en&has_verified=1&bpctr=9999999999"},
}
Status Code: 200

URL:         https://r4---sn-npoe7ne7.googlevideo.com/videoplayback?sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&signature=B754D49187CD23D2AD340227686D873154B0C283.0DB8821996F763497F09C13525D362A495CA4CA5&ip=47.88.154.139&requiressl=yes&ratebypass=yes&source=youtube&c=WEB&mn=sn-npoe7ne7%2Csn-npoeenez&mime=video%2Fmp4&id=o-AMuUfY5VzMbHNV3_l_6_4yXtSlIGhaHtf_mZqWbnYtiP&initcwndbps=1495000&ipbits=0&expire=1521570336&mm=31%2C29&fvip=4&ms=au%2Crdu&ei=wP2wWvPYDMiWqQGa04XYCA&pl=22&itag=22&mt=1521548651&dur=204.103&key=yt6&lmt=1506724426906456&mv=m
Method:      GET
Headers:     http.Header{
    "Accept-Encoding": {"gzip,deflate,sdch"},
    "Referer":         {"https://www.youtube.com/watch?v=TUk7q9zRoJI"},
    "Accept-Language": {"en-US,en;q=0.8"},
    "User-Agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"},
    "Accept":          {"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
    "Accept-Charset":  {"UTF-8,*;q=0.5"},
}
Status Code: 200

   Site:   YouTube youtube.com
  Title:   冯提莫《我想对你说baby》 "I want to say to you, baby" Feng timo Edition
   Type:   video
Quality:   hd720
   Size:   18.94 MiB (19860947 Bytes)

 0 B / 18.94 MiB [--------------------------------------------------------------------------------------------]   0.00%panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x38 pc=0x626529]

goroutine 1 [running]:
github.com/iawia002/annie/utils.FileSize(0xc04220a300, 0x55, 0xc04200bfb6)
        D:/Workplace/golang/src/github.com/iawia002/annie/utils/utils.go:43 +0x59
github.com/iawia002/annie/downloader.VideoData.urlSave(0x703f18, 0x13, 0xc042044f00, 0x51, 0xc04205c390, 0x1, 0x1, 0x12f0dd3, 0x6fc2bb, 0x5, ...)
        D:/Workplace/golang/src/github.com/iawia002/annie/downloader/downloader.go:72 +0x9b
github.com/iawia002/annie/downloader.VideoData.Download(0x703f18, 0x13, 0xc042044f00, 0x51, 0xc04205c390, 0x1, 0x1, 0x12f0dd3, 0x6fc2bb, 0x5, ...)
        D:/Workplace/golang/src/github.com/iawia002/annie/downloader/downloader.go:135 +0xb17
github.com/iawia002/annie/extractors.Youtube(0xc04200a300, 0x2b, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        D:/Workplace/golang/src/github.com/iawia002/annie/extractors/youtube.go:87 +0x5c7
main.main()
        D:/Workplace/golang/src/github.com/iawia002/annie/main.go:57 +0x416

问题应该出现在这个地方：https://github.com/iawia002/annie/blob/master/utils/utils.go#L52

Downloading YouTube playlists isn't working

I tried to download https://youtu.be/eIb1cz06UwI?list=PLoJC20gNfC2gmT_5WgwYwGMvgCjYVsIQg but it just downloads the first video. I also tested another playlist and did the same.

你好，想问一下默认存储路径在哪？

ubuntu 16.04