VPN not selectable in Web Interface

[NexusFuzzy]

VPN not selectable in Web Interface

[doomedraven]

ok thanks for headups with supervisor and fixes, i also don't recommend to use supervisor due to performance problem that i had observer in many servers, i will update it at the evening

[doomedraven]

hello, from quick check i see that everything is correct, did you restart rooter/cuckoo/web afger edit the conf? in vpn conf youhave it enabled=yes?

[NexusFuzzy]

thanks for the quick reply! Yes, I normally do sudo supervisorctl restart all after doing changes and I double checked that enabled=yes. In some of your other posts I saw that you are referring to a python script which you use to start your OpenVPN connections but I wasn't able to find it.

Could it be possible that they aren't visible because I let openvpn itsself start the tunnel at boot time?

[doomedraven]

if the tunnel is up it shouldn't affect it, the script if i understand/remember correctly it was just all in one openvpn commands

i have small suggestion instead of go with supervisor, check the log fg X so you can see maybe there is some errors in logs, bcz it looks correct so far and i didn't change code in a long time

[NexusFuzzy]

Ok I will try that. Just to be sure: The VPN should be visible in the web interface if everything works correctly?

[doomedraven]

yes, let me know if you need any help for investigation but bear in mind v1 is abondoned and we focus on v2 only https://github.com/kevoreilly/CAPEv2/

[kevoreilly]

I wouldn't say abandoned :) I am still working on it but will be shifting over to v2 soon. I will do my best to keep v1 working for another few months so that everyone has time to migrate to Py3

[kevoreilly]

Any luck with this issue?

[NexusFuzzy]

Still no luck I started from scratch, used @doomedraven 's install script to make sure I didn't miss any important dependencies, set up my VMs again, downloaded a OpenVPN config from NordVPN and started my tunnel with

sudo openvpn --config /path/to/my.config

but it still doesn't appear on the GUI

[doomedraven]

Can you share the vpn.conf? El mié., 18 dic. 2019 11:36, hariomenkel <[email protected]> escribió:

…

Still no luck I started from scratch, used @doomedraven <https://github.com/doomedraven> 's install script to make sure I didn't miss any important dependencies, set up my VMs again, downloaded a OpenVPN config from NordVPN and started my tunnel with sudo openvpn --config /path/to/my.config but it still doesn't appear on the GUI — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#461?email_source=notifications&email_token=AAOFH3Y4FGXGD45UKTX4ASTQZH4KRA5CNFSM4JS3YJWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHFVZDQ#issuecomment-566975630>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOFH3ZUNF5GTBRO5G4PNKDQZH4KRANCNFSM4JS3YJWA> .

[NexusFuzzy]

Sure:

[vpn]
enabled = yes
vpns = vpn0
description = openvpn_tunnel
interface = tun0
rt_table = tun0

My /etc/iproute2/rt_tables contains the line

400 tun0

One thing I noticed is that if I define my tun0 as "dirty line" in cuckoo.conf the change gets reflected in the UI but the VMs aren't able to connect to the internet during the analysis. I think that's part of the problem and I'll further investigate this

[doomedraven]

well what i would do, is check that here https://github.com/ctxis/CAPE/blob/master/lib/cuckoo/core/startup.py#L667 if it really loads your vpns, i have pretty the same config as you, and it works, just instead of tun0 in rt_table im using remote address

[NexusFuzzy]

I added Log statements to every crucial part of this script and everything seemed correct. Afterwards, I started an analysis this time with burp between my Browser and CAPE and after submittung the job I changed "internet" to "vpn0" and the analysis worked fine (Files were downloaded etc.)

So it seems to be a GUI issue that my VPN just doesn't get listed in the dropdown for "Network routing through dirty line or VPN"

[doomedraven]

yes but that should be the problem preload, i will check in v2, as cape1 end of life is 1.1.2020, brb to you in few mins

[doomedraven]

hm weird, i just enabled config, even without start vpn, so all data is fake, puit the same in rt_table 400 tun0

cat ../conf/vpn.conf
[vpn]
# By default we disable VPN support as it requires running utils/rooter.py as
# root next to cuckoo.py (which should run as regular user).
enabled = yes

# Comma-separated list of the available VPNs.
vpns = vpn0

[vpn0]
# Name of this VPN. The name is represented by the filepath to the
# configuration file, e.g., cuckoo would represent /etc/openvpn/cuckoo.conf
# Note that you can't assign the names "none" and "internet" as those would
# conflict with the routing section in cuckoo.conf.
name = vpn0

# The description of this VPN which will be displayed in the web interface.
# Can be used to for example describe the country where this VPN ends up.
description = openvpn_tunnel

# The tun device hardcoded for this VPN. Each VPN *must* be configured to use
# a hardcoded/persistent tun device by explicitly adding the line "dev tunX"
# to its configuration (e.g., /etc/openvpn/vpn1.conf) where X in tunX is a
# unique number between 0 and your lucky number of choice.
interface = tun0

# Routing table name/id for this VPN. If table name is used it *must* be
# added to /etc/iproute2/rt_tables as "<id> <name>" line (e.g., "201 tun0").
# ID and name must be unique across the system (refer /etc/iproute2/rt_tables
# for existing names and IDs).
rt_table = tun0

[doomedraven]

did you restart cuckoo and web after enable the conf?

[NexusFuzzy]

Weird, yes I use

sudo supervisorctl restart all

and according to the logs cuckoo etc. get restarted. Is there anything I need to add in cuckoo.conf to pick up the VPNs?

[doomedraven]

no, nothing, i only touched vpn.conf, and restarted web and cuckoo, thats all

[NexusFuzzy]

I guess I'll close that issue as not reproducable and once I found the root cause I'll update this issue. Anyways, thanks a ton for your support and the effort you are putting in this project!

[doomedraven]

let us know if you solve it, but if it not listed for some reason it has some problem with load data where i pointed you https://github.com/ctxis/CAPE/blob/master/lib/cuckoo/core/startup.py#L667

i hope you can fix it asap

[NexusFuzzy]

Okay looks like I found the problem!

The file CAPE/web/submissions/views.py seems to be the problem:

In line 29

from lib.cuckoo.core.rooter import vpns, _load_socks5_operational

but vpns don't seem to get filled with items somewhere (at least in my case).

In CAPE/web/web/settings.py where te list of VPNs seem to get populated, there is no reference to rooter either instead it's loading

vpn = Config("vpn")

instead. So what did I do to get the list ov VPNs?

I edited views.py and added the following code:

vpn = Config("vpn")

vpn_list = []
if vpn.vpn.enabled:
for name in vpn.vpn.vpns.split(","):
    name = name.strip()
    if not name:
        continue
    entry = vpn.get(name)
    vpn.vpn[entry.name] = entry
    vpn_list.append(entry)

and a little below at "return render" I added "vpns": vpn_list

Not sure if this is the right way to fix it but at least in my case it worked.

[doomedraven]

well as you saw, it all works out of the box when i have tested, so that shouldn't be a global problem

im double checking you reporting code

grep -r vpns .
./web/templates/submission/index.html:                                        {% for vpn in vpns %}
./web/submission/views.py:from lib.cuckoo.core.rooter import vpns, _load_socks5_operational
./web/submission/views.py:                       "vpns": list(vpns.values()),
./utils/vpncheck.py:from lib.cuckoo.core.rooter import rooter, vpns
./utils/vpncheck.py:        if vpn not in vpns:
./utils/vpncheck.py:        if not rooter("nic_available", vpns[vpn].interface):
./utils/vpncheck.py:        ipaddr = get_ip_address(vpns[vpn].interface)
./utils/vpncheck.py:        rooter("forward_enable", vpns[vpn].interface, vpns[vpn].interface, ipaddr)
./utils/vpncheck.py:        rooter("srcroute_enable", vpns[vpn].rt_table, ipaddr)
./utils/vpncheck.py:                print(vpns[vpn].name, ipaddr, ret)
./utils/vpncheck.py:        rooter("forward_disable", vpns[vpn].interface, vpns[vpn].interface, ipaddr)
./utils/vpncheck.py:        rooter("srcroute_disable", vpns[vpn].rt_table, ipaddr)
./lib/cuckoo/core/startup.py:from lib.cuckoo.core.rooter import rooter, vpns, socks5s
./lib/cuckoo/core/startup.py:    # available through the vpns variable. Also enable NAT on each interface.
./lib/cuckoo/core/startup.py:        for name in routing.vpn.vpns.split(","):
./lib/cuckoo/core/startup.py:            vpns[entry.name] = entry
./lib/cuckoo/core/startup.py:        if routing.routing.route not in vpns and routing.routing.route not in socks5s:
./lib/cuckoo/core/rooter.py:vpns = dict()
./lib/cuckoo/core/scheduler.py:from lib.cuckoo.core.rooter import rooter, vpns, _load_socks5_operational
./lib/cuckoo/core/scheduler.py:        elif self.route in vpns:
./lib/cuckoo/core/scheduler.py:            self.interface = vpns[self.route].interface
./lib/cuckoo/core/scheduler.py:            self.rt_table = vpns[self.route].rt_table
./lib/cuckoo/core/scheduler.py:        if self.route in vpns:
./lib/cuckoo/core/scheduler.py:            for vpn in vpns.values():

as you can see that is populated in startup.py, the same for socks5s

so that is just a way to pass the variable as we verify that vpn is really up

[doomedraven]

╰─± grep -r init_routing .
./cuckoo.py:    from lib.cuckoo.core.startup import init_rooter, init_routing
./cuckoo.py:    init_routing()
./web/web/settings.py:from lib.cuckoo.core.startup import init_rooter, init_routing
./web/web/settings.py:init_routing() <------- this populates vpns for webgui
./utils/vpncheck.py:from lib.cuckoo.core.startup import init_rooter, init_routing
./utils/vpncheck.py:    init_routing()
./lib/cuckoo/core/startup.py:def init_routing():

this is what populating that fields

[doomedraven]

ah yap well you using very old repo and depricated :D go to https://github.com/kevoreilly/CAPEv2/ for current code and updated, as we don't support capev1 anymore

[NexusFuzzy]

Okay I'll set up a new machine with CAPEv2 and see if it's running with the new one

[doomedraven]

there are tons of fixes/improvements and features in v2, sincerely i wouldn't even spend anymore 1second on v1, if you don't want to fight with all what was fixed in v2

[NexusFuzzy]

So after a bit of problems with the capev2.sh script I managed to get it running and the VPN is there in the list. So, if someone else is stumbling across this problem: Do yourself a favour and update to CAPEv2 :-) Thanks @doomedraven !

[doomedraven]

if you report me what problem you had with cape2.sh i will fix that :)

[NexusFuzzy]

There were a few things:

• Missing Dependencies: curl, git python3-pip (I think they were for yara installation)
• The supervisor part is using "python" to replace text but on a 18.04 you normally have no Python 2.7
• On the logrotate part I had to remove the comment symbols on one part and comment out the other part

I'll post a fixed version of the script tomorrow which worked on a freshly installed Ubuntu 18.04 Desktop.

Thanks for all the work you guys put into Cape! Much appreciated!

[doomedraven]

cool thanks :)

[NexusFuzzy]

I attached the fixed file.

What I changed is the following:

• Added curl and git to Dependencies
• Switched the comment part in Logrotate
• Added "-y" to libpcre installation in mongo section
• Changed the text replacement part of supervisor to use python3 and to not read/write binary
• Fixed malheur installation. I have the feeling that the version number changes from time to time which made the script break so I added a "find" to get the correct filename for the dpkg

I tested it on a freshly installed Ubuntu 18.04 Desktop and it worked flawless. Just one thing: If you select "all" the supervisor part of the script doesn't get called so I don't know if this is intentionally or not. So after running

sudo ./cape all cape 192.168.178.123 | tee cape.log

I had to run

sudo ./cape supervisor cape 192.168.178.123 | tee cape.log

cape2.txt

[doomedraven]

also about the comment section, what command line interpreter do you use bash/zsh? i have tested that for commands and it was working fine

[NexusFuzzy]

You mean the logrotate part? I was receiving the error

duplicate entry for alternatives.log

from logrotate itsself

[doomedraven]

no, was in general, cool thanks again foro update

[NexusFuzzy]

I am using Python 3.6.9

[doomedraven]

ah i saw you uncommented malheur, malheur is "dead" i don't even updated code for it, as when you really got decent pack of samples analyzed, it will be so damn slow that you will disable it, i will just remove it from cape2.sh, i left it there jsut for people but it think is time to get full rid of it in cape too

about supervisor logrotate i have disabled it as it was giving some problems i don't remember which exactly

the rest is added, thank you again