fleetdm/fleet

Open-source platform for IT, security, and infrastructure teams. (Linux, macOS, Chrome, Windows, cloud, data center)

News   ·   Report a bug   ·   Handbook   ·   Why open source?   ·   Art

Open-source platform for IT and security teams with thousands of computers. Designed for APIs, GitOps, webhooks, YAML, and humans.

What's it for?

Organizations like Fastly and Gusto use Fleet for vulnerability reporting, detection engineering, device management (MDM), device health monitoring, posture-based access control, managing unused software licenses, and more.

Explore data

To see what kind of data you can use Fleet to gather, check out the table reference documentation.

Out-of-the-box policies

Fleet includes out-of-the box support for all CIS benchmarks for macOS and Windows, as well as many simpler queries.

Take as much or as little as you need for your organization.

Supported platforms

Here are the platforms Fleet currently supports:

• Linux (all distros)
• macOS
• Windows
• Chromebooks
• Amazon Web Services (AWS)
• Google Cloud (GCP)
• Azure (Microsoft cloud)
• Data centers
• Containers (kube, etc)
• Linux-based IoT devices

Lighter than air

Fleet is lightweight and modular. You can use it for security without using it for MDM, and vice versa. You can turn off features you are not using.

Openness

Fleet is dedicated to flexibility, accessibility, and clarity. We think everyone can contribute and that tools should be as easy as possible for everyone to understand.

Good neighbors

Fleet has no ambition to replace all of your other tools. (Though it might replace some, if you want it to.) Ready-to-use, enterprise-friendly integrations exist for Snowflake, Splunk, GitHub Actions, Vanta, Elastic Jira, Zendesk, and more.

Fleet plays well with Munki, Chef, Puppet, and Ansible, as well as with security tools like Crowdstrike and SentinelOne. For example, you can use the free version of Fleet to quickly report on what hosts are actually running your EDR agent.

While most folks prefer to use one or the other, Fleet can also coexist peacefully with Rapid7 and other agent-based vulnerability scanners. This can be useful during migrations.

Free as in free

The free version of Fleet will always be free. Fleet is independently backed and actively maintained with the help of many amazing contributors.

Longevity

The company behind Fleet is founded (and majority-owned) by true believers in open source. The company's business model is influenced by GitLab (NYSE: GTLB), with great investors, happy customers, and the capacity to become profitable at any time.

In keeping with Fleet's value of openness, Fleet Device Management's company handbook is public and open source. You can read about the history of Fleet and osquery and our commitment to improving the product.

Is it any good?

Fleet is used in production by IT and security teams with thousands of laptops and servers. Many deployments support tens of thousands of hosts, and a few large organizations manage deployments as large as 400,000+ hosts.

Chat

Please join us in MacAdmins Slack or in osquery Slack.

The Fleet community is full of kind and helpful people. Whether or not you are a paying customer, if you need help, just ask.

Contributing          

The landscape of cybersecurity and IT is too complex. Let's open it up.

Contributions are welcome, whether you answer questions on Slack / GitHub / StackOverflow / LinkedIn / Twitter, improve the documentation or website, write a tutorial, give a talk at a conference or local meetup, give an interview on a podcast, troubleshoot reported issues, or submit a patch. The Fleet code of conduct is on GitHub.

What's next?

To see what Fleet can do, head over to fleetdm.com and try it out for yourself, grab time with one of the maintainers to discuss, or visit the docs and roll it out to your organization.

Production deployment

Fleet is simple enough to spin up for yourself. Or you can have us host it for you. Premium features are available either way.

Documentation

Complete documentation for Fleet can be found at https://fleetdm.com/docs.

License

The free version of Fleet is available under the MIT license. The commercial license is also designed to allow contributions to paid features for users whose employment agreements allow them to contribute to open source projects. (See LICENSE.md for details.)

Fleet is built on osquery, nanoMDM, Nudge, and swiftDialog.

fleet's Issues

Welcoming Andrew Bare

Note: This template is a work in progress adapted from a different team. I'll update the template as we make progress.

Welcome to Fleet! This issue tracks TODOs required for all new teammates for you and your manager to help welcome you efficiently to the team.

Manager TODOS

• Create the #hiring- channel for this teammate in Slack.
• Confirm System Administrator-created accounts in all of the required systems (see list below)
• If a teammate is non-US based, email the Operations group with the teammate's name, start date, and a reminder to pre-generate the first 12 invoices (if applicable) Set up teammate in Gusto (if US based) or Pilot (otherwise)
• Schedule a 30-minute all-team "welcome" meeting
• Create new 1:1 agenda doc for teammate based on GitLab's 1:1 template (i.e. copy one of the existing 1:1 agendas)
• Schedule recurring daily check-ins with teammate for the first 2 weeks
• Schedule a recurring 1:1 at starting in week 3

New teammate TODOS

Administrative

• Ensure your payroll and employment information is correct and up to date, reach out to Mike McNeil if not.
• Accept the invite to all of our productivity tools

• Set a consistent picture as your avatar in all of our collaboration tools (GitHub, Slack, GSuite, etc.)
• Update your LinkedIn profile and send connection requests to your colleagues. (This is a suggestion, not a requirement. Consider using the same picture as your linkedin everywhere at Fleet for continuity.)

• Confirm that your GitHub notifications are on and that you are able to receive them

Get to know the company

• Read through handbook/ Google Drive
• Ask @mikermcneil about our other resources for new team members

System administrator TODOs

For the system administrator to complete / teammate to confirm.

• Google Workspace (email address) and appropriate group membership
• [OnePassword]
  • Add to vaults (after the invite is accepted)
• https://app.orbit.love
• Trello
• Fleet Slack
• osquery Slack
• GCal for Slack (TODO: replace w/ Reclaim's Slack app when new release is ready)
• Reclaim.ai for syncing personal availability
• Zoom (if applicable) - use "Basic" account for engineers, "Licensed" account for teammates who we expect to host meetings frequently

• [YouTube] - use "Manager" permission level
• Add to Google Analytics (collaborate + read + analyze permissions)
• Local Fleet (QA-able, demo-able, update-able)
  • Node
  • fleetctl
  • Docker Desktop
  • fleetctl preview
  • Simulated hosts (containers running osqueryd)

Reference: Additional setup for engineers

• Setup local development environment
  • Fleet (for contributors)
  • MySQL database
  • Redis

User avatar no longer clickable for menu

What version of fleet are you using (fleet version --full)?

master (This has not made it to a release)

If this is a UI issue: What browser are you using?

Chrome & Firefox on macOS

What did you do?

Click the user name/avatar

What did you expect to see?

Pop up menu with user options

Like this

What did you see instead?

No apparent UI response

Following README instructions fails with error when attempting to write to `/logs`

What version of fleet are you using (fleet version --full)?

🐍 fleetctl -v
fleetctl - version 3.6.0
  branch: 	master
  revision: 	68718c183fe9288022ab84e4ca8e4439f07f7e14
  build date: 	2021-01-07T22:31:26Z
  build user: 	noahtalerman
  go version: 	go1.15.3

What operating system are you using?

macOS

What did you do?

Followed the "Try Fleet" instructions in the README

What did you expect to see?

A working demo of Fleet

What did you see instead?

When trying to write to /logs, a directory which won't exist for most users, fails w/ a permission error

Automate the setup process and the fleetctl configuration for the fleetctl preview experience.

The fleetctl preview command is a slick tool to help new visitors/users to check out Fleet quickly.

Currently, a user of the fleetctl preview tool will see the setup flow when they first navigate to their local Fleet instance in the browser.
Also, the same user will have to manually set their fleetctl configuration to utilize other fleetctl commands.

The goal is to simplify the fleetctl preview experience even more. We make the assumption that users who want to quickly see what Fleet has to offer could be slowed down (or confused) by having to go through the setup and configuration processes.

The specific solution discussed for automating the fleetctl configuration is to have the preview command automatically set a config context with the name preview. The command will also set the default context if it isn't already.
A potential solution for automating the setup experience is to create and log in the admin user. The log in credentials can then be present in the terminal output. Example:

• Username: John Appleseed
• Email: [email protected]
• Password: fleetpreview123!

[osquery-in-a-box] Can't see any data inside Kibana.

Hi there,
Thank you for making the setup easier with this GIT.

I followed your detailed guidelines to the letter and i can see all hosts online inside fleet.
I can't see any data inside kibana though (i run 2-3 queries across the whole fleet successfully).
"Check for new data" always results in nothing.

Should i configure something inside the settings on Kibana?
Since i don't see any data, i can't configure the index name...

Thanks in advance for the help...

Update schema documentation to 4.6.0

Query editor UI renders strangely in desktop Safari

What version of fleet are you using (fleet version --full)?

3.4.0

What browser are you using?

Safari (13.1.1) on macOS

What did you see?

Originally tracked in kolide/fleet#1324

Latest update from @Rylon:

I…have this issue in Safari (13.1.1) on macOS Catalina 10.15.5, but it seems to be fixed in Firefox (79)

Publish Fleet's H1 2021 roadmap

This issue's goal is to provide a consistent location for the discussion of Fleet's future use cases, features, and solutions.

We're very interested in what the community and potential customers are looking for. Please feel free to add any suggestions or thoughts in the comments!

Listed below include some informal thoughts that we think will benefit folks:

• One agent to rule them all. We want to provide (or allow users to build) any security/it/compliance monitoring they need by deploying a single agent (osquery).
• Build alerting and enrichment into Fleet so that queries can be defined along with the alerts and enrichment (thinking to evolve the yaml format to support some of these things).
• Make it easier to deploy and update osquery, and extensions.
• Build a datastore tailored to storing information logged from osquery (not competing with Splunk/ELK for longer-term storage, but making the management and querying of "facts" available from osquery more efficient).
• Fine-grained authorization that allows exposing the capabilities of Fleet to more of the organization without compromising security/stability.
• Privacy and user respect is important
• Expose a UX for endpoint users to understand how/what data is being collected from their devices.

Make spec requests and responses use consistent schema

Example request:

{
  "spec": {
    "secrets": [
      {
        "name": "default",
        "secret": "s96I5x42hhN6c/kqxnpnX2ODkVJBVrOP",
        "active": true,
        "created_at": "2020-12-24T15:57:49Z"
      }
    ]
  }
}

Example response:

{
  "specs": {
    "secrets": [
      {
        "name": "default",
        "secret": "s96I5x42hhN6c/kqxnpnX2ODkVJBVrOP",
        "active": true,
        "created_at": "2020-12-24T15:57:49Z"
      }
    ]
  }
}

It would be nice if thy both used spec or specs for consistency. This is an API-breaking change and needs to be saved for the major version release.

Incorrect docs link within UI

Add New Host modal has external link labeled "Add Hosts Documentation " at the top linked to https://github.com/fleetdm/fleet/blob/master/docs/infrastructure/adding-hosts-to-fleet.md which is not a valid docs file.

I believe it is intended to direct to https://github.com/fleetdm/fleet/blob/master/docs/2-Deployment/3-Adding-hosts.md

Appears this is in :

fleet/frontend/components/hosts/AddHostModal/AddHostModal.jsx

Line 112 in 49e71e4

href="https://github.com/fleetdm/fleet/blob/master/docs/infrastructure/adding-hosts-to-fleet.md"

Looks like there is most likely a handful of these changes lurking: https://github.com/fleetdm/fleet/search?q=master%2Fdocs

Descriptions of saved queries need a little less social distancing when the text wraps (/queries/manage)

What version of fleet are you using (fleet version --full)?

🐍 ./build/fleet version --full
fleet - version 3.5.1-4-ge9a77cc6
  branch: 	master
  revision: 	e9a77cc64a0525c2cb0ad9542b3617cf566efc1a
  build date: 	2020-12-17T19:27:05Z
  build user: 	mikermcneil
  go version: 	go1.15.3

What operating system are you using?

[email protected]

If this is a UI issue: What browser are you using?

What did you do?

https://localhost:8080/queries/manage

What did you expect to see?

In the description, either less line height or ellipses-style overflow.

What did you see instead?

fleetctl - error setting up Fleet: '@' character not allowed in usernames

What version of fleet are you using (fleet version --full)?

fleet - version 3.3.0
branch: master
revision: f93a952
build date: 2020-11-05T06:02:25Z
build user: zwass
go version: go1.15

What operating system are you using?

Alpine docker container

What did you do?

fleetctl setup --email [email protected] --password toor

What did you expect to see?

That the user was created and Fleet setup.

What did you see instead?

No username supplied, using email as username
error setting up Fleet: setup received status 422 Validation Failed: '@' character not allowed in usernames

This has worked fine on previous releases.

[osquery-in-a-box] Windows Support

Will it support this information on windows platform?
https://github.com/fleetdm/osquery-in-a-box

Cannot add same query to pack more than once (results in 5xx)

• This is a duplicate of kolide/fleet#2202.
• The issue was discussed during Roundup (12/2/2020).
• Noah has verified that the same error behavior regardless of the values chosen for the configuration fields.
• Error message displayed: inserting scheduled query: Error 1062: Duplicate entry 'query-1' for key 'unique_names_in_packs'

What version of fleet are you using (fleet version --full)?

2.4.0

What operating system are you using?

Docker image: kolide/fleet:2.4.0

What did you do?

1. create query 'my_awesome_query'
2. create pack 'my_pack'
3. (through the UI) add 'my_awesome_query' to pack, with snapshot false and interval 3600
4. (through the UI) add 'my_awesome_query' to pack again, with snapshot true, and interval 43200

What did you expect to see?

Query added to pack twice with different schedules / intervals.

Alternately: 400 error page explaining that adding a query to a pack more than once is not supported. I don't think a 5xx response is correct.

What did you see instead?

5xx error page

Possibly missing osquery client logs

Hello!

Please help to solve my problem.

We try to use fleet and deploy clients to 1673 hosts.

Our fleet config is:

spec:
config:
decorators:
always:
- >-
SELECT version as kaspersky_version from programs where name like
"%kaspersky%" and name NOT LIKE "%Плагин%" and name NOT LIKE "%агент%"
and name NOT LIKE "%админ%" and name NOT LIKE "%Center%" and name NOT
LIKE "%Management%" and name NOT LIKE "%Connection%";
- >-
SELECT version as dallas_version from programs where name like
"Dallas%";
- >-
SELECT version as yandex_version from programs where name like
"Yandex";
- SELECT codename FROM os_version;
- >-
SELECT address AS endpoint_ip1 FROM interface_addresses where address
like '10.%';
- >-
SELECT address AS endpoint_ip2 FROM interface_addresses where address
not like '%:%' and address not like '127%' and address not like '169%'
and address not like '10.%' order by interface asc limit 1;
- SELECT hardware_serial FROM system_info;
- SELECT hostname AS hostname FROM system_info;
- >-
SELECT user AS logged_user FROM logged_in_users WHERE user <> '' ORDER
BY time LIMIT 1
options:
decorations_top_level: true
disable_distributed: false
distributed_interval: 10
distributed_plugin: tls
distributed_tls_max_attempts: 3
distributed_tls_read_endpoint: /api/v1/osquery/distributed/read
distributed_tls_write_endpoint: /api/v1/osquery/distributed/write
enable_windows_events_publisher: true
enable_windows_events_subscriber: true
logger_plugin: tls
logger_tls_endpoint: /api/v1/osquery/log
logger_tls_period: 10
pack_delimiter: _
overrides: {}

We have 3 packs with 85 queries. Some queries only to get statistic.

We have problems with using packs.
We have got logs only from 262 hosts.

osquery_status log contains a lot of strings like:

{"s":1,"f":"events.cpp","i":311,"m":"Expiring events for subscriber: windows_events (overflowed limit 50000)","h":"69FFFE64-8784-2C57-D2B6-7824AFA14430","c":"Thu Dec 10 13:10:10 2020 UTC","u":1607605810}
{"s":1,"f":"events.cpp","i":311,"m":"Expiring events for subscriber: windows_events (overflowed limit 50000)","h":"69FFFE64-8784-2C57-D2B6-7824AFA14430","c":"Thu Dec 10 13:10:25 2020 UTC","u":1607605825}
{"s":1,"f":"events.cpp","i":311,"m":"Expiring events for subscriber: windows_events (overflowed limit 50000)","h":"69FFFE64-8784-2C57-D2B6-7824AFA14430","c":"Thu Dec 10 13:10:34 2020 UTC","u":1607605834}
{"s":1,"f":"events.cpp","i":311,"m":"Expiring events for subscriber: windows_events (overflowed limit 50000)","h":"69FFFE64-8784-2C57-D2B6-7824AFA14430","c":"Thu Dec 10 13:10:46 2020 UTC","u":1607605846}
{"s":1,"f":"events.cpp","i":311,"m":"Expiring events for subscriber: windows_events (overflowed limit 50000)","h":"69FFFE64-8784-2C57-D2B6-7824AFA14430","c":"Thu Dec 10 13:10:58 2020 UTC","u":1607605858}
{"s":1,"f":"events.cpp","i":311,"m":"Expiring events for subscriber: windows_events (overflowed limit 50000)","h":"69FFFE64-8784-2C57-D2B6-7824AFA14430","c":"Thu Dec 10 13:11:11 2020 UTC","u":1607605871}
{"s":1,"f":"events.cpp","i":311,"m":"Expiring events for subscriber: windows_events (overflowed limit 50000)","h":"69FFFE64-8784-2C57-D2B6-7824AFA14430","c":"Thu Dec 10 13:11:26 2020 UTC","u":1607605886}
{"s":1,"f":"events.cpp","i":311,"m":"Expiring events for subscriber: windows_events (overflowed limit 50000)","h":"69FFFE64-8784-2C57-D2B6-7824AFA14430","c":"Thu Dec 10 13:11:38 2020 UTC","u":1607605898}

and:

{"s":0,"f":"processes.cpp","i":379,"m":"Failed to get cwd for 4 with 232","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":365,"m":"Failed to lookup path information for process 4","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":337,"m":"Failed to get PEB UPP for 4 with 0","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":379,"m":"Failed to get cwd for 4 with 232","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":365,"m":"Failed to lookup path information for process 4","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":337,"m":"Failed to get PEB UPP for 4 with 0","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":379,"m":"Failed to get cwd for 4 with 232","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":365,"m":"Failed to lookup path information for process 4","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":337,"m":"Failed to get PEB UPP for 4 with 0","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":379,"m":"Failed to get cwd for 4 with 232","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":337,"m":"Failed to get PEB UPP for 612 with 5","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":379,"m":"Failed to get cwd for 612 with 232","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":337,"m":"Failed to get PEB UPP for 612 with 5","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":379,"m":"Failed to get cwd for 612 with 232","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}
{"s":0,"f":"processes.cpp","i":337,"m":"Failed to get PEB UPP for 632 with 5","h":"f577f0c6-68f0-4ea1-ac35-f5995cbdfbf5","c":"Thu Dec 10 22:18:26 2020 UTC","u":1607638706}

Remove deprecated configuration options

There are some deprecated configuration options with warning messages in serve.go that should be removed for the next major version release.

Make UX snappier when targeting multiple, particular hosts via search

when selecting hosts to run a query against you have to search each time for a host and select it.
Ex:
assume you have 10 php hosts, which you want to run a query against.
each time you will search for php word to select the first host, then the second, and so on till you select the 10 hosts.
which means you are running many queries on the DB and those queries take time(long time).

Suggestion to have those options for the returned results:

• select all option
• deselect all option
  

Hosts UI

Goals

This project is about adding a new "host details" page and some oft-requested features for browsing hosts.

Steps

• I. Drafting
  • High-level goals, due date set
  • GitHub issue
  • Slack channel (#2021-…)
  • Slack channel topic set (🏁 Jan 31: https://github.com/fleetdm/fleet/issues/TODO)
  • Brainstorming, design research, talking to users
  • Lo-fi wireframes (personal draft in Figma)
    • Vision review w/ @edamamedesign, directional/aesthetic revisions
    • Practical review w/ @mikermcneil, simplifying/UX revisions
  • Hi-fi wireframes (shared project in Figma)
    • Engineering review w/ @zachW, tactical revisions
  • Final wireframes
    • Shared as a comment in this issue
    • https://www.instagram.com/fleetctl
• II. Implementation
  • Pull request(s)
    • Code changes
    • Documentation changes
    • Short Loom video demo (the happy path + the edge cases you've tested)
    • Engineering review, quality/style revisions
• III. Release
  • Release tagged
  • Release notes written
  • Upgrade considerations documented (e.g. breaking change, DB schema migrations, etc)
  • Release announced
    • Release published
      • GitHub
      • NPM
      • Docker
    • Release blog post published
    • Release announced in osquery Slack
    • Release announced on social media
      • https://twitter.com/fleetctl
      • https://linkedin.com/company/fleetdm
      • https://www.facebook.com/fleetdm
      • https://www.instagram.com/fleetctl/

UI: Host listing doesn't expand to fit window on Chrome

What version of fleet are you using (fleet version --full)?

fleet - version 3.5.0
  branch: 	master
  revision: 	4c27a6786a0ad78cd52aeb55fb7f07502053695d
  build date: 	2020-12-10T23:10:49Z
  build user: 	noahtalerman
  go version: 	go1.15.3

What operating system are you using?

MacOS 11.1 (Fleet is hosted on Ubuntu 18.04)

If this is a UI issue: What browser are you using?

Chrome v87

What did you see instead?

Missing image assets

It looks like some assets are missing from the latest release / master branch.

If you go to the new query page and click on an asset some images fail to load.

So far I've found the following assets missing:

/assets/images/[email protected]
assets/images/[email protected]

I'm assuming these are simply missing image assets from: https://github.com/fleetdm/fleet/tree/master/assets/images

Possible ways to detect missing image assets in tests

Problem

The addition of the new <Icon /> component resulted in missing image assets in Fleet's UI. Discovered in #168.

Goal

Get tests to detect missing assets.

Current approaches (1)

• In the unit tests for each icon component, import all image assets that the respective components are expected to render. Group the expected assets in an object and use Jest's expect().toHaveProperty()to test the <img />'s src property against the object.

Make "name" and "secret" required when creating enroll secret

Sending a POST request with the /spec/enroll_secret API endpoint allows me to send a valid request with no name or secret property specified.
This creates an unnamed and empty enroll secret.

Example request body

{
    "spec": {
        "secrets": [
            {
                "active": true
            }
        ]
    }
}

Enroll secrets table

Carve Files to Remote Destination

Currently Carving is saving the file to Fleet DB.

a proposed option, Fleet could have the option to save carved file to :

• Remote Volume, like Fleet is on K8s and could have access to a volume dedicated just for carving
• Cloud Bucket like GCP, AWS and Azure
• Remote SFTP Destination

this would be very helpful when you do incident response and want to carve Malware/File from a host.
DB have a lots of limitation and sizing.

Hoping you could consider that option.

`fleetctl preview` spits out `fleet-preview/` folders

Currently, fleetctl preview creates a fleet-preview/ folder in the current working directory.

Could this go somewhere else, globally? Maybe it could be created in a .tmp/ folder in the directory of the NPM package itself.

There's an argument to be made for having fleetctl preview create this directory, so you could preview more than one Fleet demo at the same time. Which is cool, but not super necessary, since it's just a demo- and the cons probably outweigh the pros.

Since Docker runs globally on the system, my expectation would be that the files related to osquery-in-a-box should exist in one place, to avoid potential confusion about which Fleet is which, and which one is being started up in Docker.

i.e. if I move or delete the fleet-preview/ folder -- and/or if I run fleetctl preview again from a different pwd

🐍 fleetctl -v
fleetctl - version 3.6.0
  branch: 	master
  revision: 	68718c183fe9288022ab84e4ca8e4439f07f7e14
  build date: 	2021-01-07T22:31:26Z
  build user: 	noahtalerman
  go version: 	go1.15.3

How often do labels refresh? Is the refresh frequency configurable?

I have around 10 custom labels for my fleet, and I am trying to understand the performance impact of having more labels on the system.

Add --debug flag to fleetctl

This flag would cause fleetctl to log all requests and responses.

Possibly using https://github.com/motemen/go-loghttp?

Filters in Host page

1) What version of Fleet are you using (fleet version --full)?
fleet - version 3.5.1
branch: master
revision: 55a2aa2
build date: 2020-12-15T02:50:05Z
build user: zwass
go version: go1.15

What operating system are you using?
NAME="CentOS Linux"
VERSION="8"

What did you do?
Opened Fleet GUI, Hosts page

What did you expect to happen?
I wanted to quickly find newly enrolled host. His hostname, osquery version, uptime, etc.

What happened instead?
Given that there can be some variation in the hostname, it becomes difficult to find it quickly.
There is pagination in this page, but there is no way to select the number of hosts per page.
I have to go to the Queries page and alternately try to enter the host name until I find the correct one, and then make requests to it to get the information described above.

Enhancement details
It would be great if the Hosts page had filters for each field, similar to those that appear in the query results via GUI. It would be possible to quickly target our host and, if necessary, quickly navigate to it and execute the necessary queries.

Complete documentation for Fleet API (all endpoints)

This issue describes the project of documenting all Fleet API endpoints. This google doc contains a list of all completed and to be completed endpoints.

Title, description, parameters, and example code will be added for each endpoint.

Create Labels on Fleet DB

Labels are really useful to segment and categories Hosts. but labels are queries that run on each system.
it make more sense to have to kinds of labels:

• Normal Query Labels (Currently used)
• Labels on Fleet DB (Data collected by Fleet)

Examples:
when you want to create a hosts Labels, get all hosts that have "%.dev.%" this will run on all hosts. but it make more sense to just run on DB.
SELECT * FROM system_info where hostname like '%.dev.%';
basically any information that is known to be in the DB, it make more sense to run that label on the DB and not query all hosts.

Handle MySQL password for local development environment

What version of fleet are you using (fleet version --full)?

fleet - version 3.5.1-24-g72637d64
branch: master
revision: 72637d6
build date: 2021-01-04T20:31:51Z
build user: noahtalerman
go version: go1.15.3

What operating system are you using?

macOS Big Sur Version 11.0.1

What did you do?

I pulled the latest changes from master.
Then, I removed the MySQL docker volume for my local instance by running docker volume rm fleet-1_mysql-persistent-volume
Then I ran docker-compose up.
Then I ran make deps, make generate-dev, and make.
Then I ran build/fleet serve --auth_jwt_key=insecure.

What did you expect to see?

I expected to see a message in my terminal that would instruct me to rebuild my database with build/fleet prepare db

What did you see instead?

ts=2021-01-04T20:38:26.799484Z mysql="could not connect to db: Error 1045: Access denied for user 'kolide'@'172.26.0.1' (using password: NO), sleeping 0s"

Summary

I suspect the changes in #141 may have introduced this scenario. Tracking it for now and will dig deeper at end of day today.

Unexpected migration warning message when upgrading to 3.3.0

What version of fleet are you using (fleet version --full)?

3.3.0

What operating system are you using?

Any

What did you do?

Start the Fleet server

What did you expect to see?

No errors/warnings

What did you see instead?

################################################################################
# WARNING:
#   Your Fleet database is missing required migrations. This is likely to cause
#   errors in Fleet.
#
#   Run `./build/fleet prepare db` to perform migrations.
################################################################################

This looks to be an issue that would be experienced by any user upgrading from a previous version of Fleet to 3.3.0. A new install on 3.3.0 would not trigger the warning.

There should not actually be any problem for this upgrade scenario besides the user seeing the warning message.

This is caused by the removal of data migrations in kolide/fleet#2327. It will be solved by adding back an empty migration with a filename matching the timestamp from https://github.com/kolide/fleet/pull/2327/files#diff-5eb1f9707277bc63ef0142ec37ecc41af77bfcea05af27237822f81f99200852.

New UI images broken with server prefix

Testing out FleetDM 3.6, using the a prefix of /fleet (KOLIDE_SERVER_URL_PREFIX=/fleet)

The images are 404, as they do not take into account the prefix.

Fleet server memory issues (associated with duplicate host enrollment)

What version of fleet are you using (fleet version --full)?

Up to latest (3.4.0)

What operating system are you using?

Any

If this is a performance issue: Please attach the debug archive.

If any user is encountering this, please help us get a debug archive.

What did you do?

Users seem to encounter this issue when they have many hosts deployed with the same hardware UUID and osquery is configured with --host_identifier=uuid. This is likely due to a cloned VM image with the UUID not changed.

What did you expect to see?

No issues with the Fleet server.

What did you see instead?

Fleet server exhausts resources (particularly RAM) as the enrollments begin overriding each other.

Note: This scenario indicates a misconfiguration of the osquery deployment as multiple different hosts identify as the same host to the Fleet server. Operators will need to fix the misconfiguration, but we can and should fix Fleet to prevent the whole server from falling over.

'Online' status icon in hosts list isn't rendering.

What version of fleet are you using (fleet version --full)?

fleet - version 3.5.0-3-gdc2befaa
branch: master
revision: dc2befa
build date: 2020-12-14T17:54:46Z
build user: noahtalerman
go version: go1.15.3

What operating system are you using?

macOS BigSur

If this is a UI issue: What browser are you using?

Chrome

What did you do?

Navigated to the Hosts page

What did you expect to see?

Expected to see green, checkmark 'online' icons in the hosts list.

What did you see instead?

Summary

This bug was introduced in #128.
We're working on finalizing new 'online' and 'offline' icons for the status column.

Change extension fleetctl-windows

What version of fleet are you using (fleet version --full)?

3.4.0

What operating system are you using?

Windows

What did you do?

Tried to unpack fleetctl-windows.tar.gz

What did you expect to see?

Unpacked fleetctl-windows files

What did you see instead?

Error unpacking as Windows does not natively support tar.gz

Request:
Change the extension of fleetctl-windows.tar.gz to .zip as if you want to use it on Windows .zip is natively supported for unpacking.

Expose live query error information in UI

Goal

Provide Fleet users more information for debugging when a live query fails on a particular host by surfacing error information returned from osquery.

Include the complete implementation in release of Fleet scheduled for end of Jan.

Steps

• @zwass populates the backend with error from osquery
• @noahtalerman exposes errors from osquery in the Fleet UI

What the user sees now (failed query run on one host)

Kubernete deployment about Kolide Fleet Server on AWS

My ex-team member followed the below link recommendation about using Kubernetes to deploy Kolide Fleet web server:
https://github.com/kolide/fleet/blob/master/docs/infrastructure/fleet-on-kubernetes.md

He used these 3 below files, to successfully deployed Kolide Fleet web server on AWS and the Osquery result output on server side is 'file system' (file location: /tmp/osquery_result):

https://github.com/kolide/fleet/blob/master/examples/kubernetes/fleet-migrations.yml,
https://github.com/kolide/fleet/blob/master/examples/kubernetes/fleet-deployment.yml,
https://github.com/kolide/fleet/blob/master/examples/kubernetes/fleet-service.yml.

However, his ex-team member already left my company. Now, I am ready to change the Osquery result output from file system to others (kinesis, firehose, redis, pubsub, etc.) on the server side, and make redeployment about Kolide fleet server. However, when I just run the original 'fleet-deployment.yml' file, with the command 'kubectl apply -f fleet-deployment.yaml -n fleet', then the kolide fleet server will crash with status '500' error. When I retrieve the log history from the Kolide fleet server pod, then the error message is something like below:

"2020/12/09 18:42:43 http: TLS handshake error from 10.0.2.54:62379: EOF
2020/12/09 18:42:43 http: TLS handshake error from 10.0.15.33:64346: EOF
2020/12/09 18:42:43 http: TLS handshake error from 10.0.15.33:17935: EOF
ts=2020-12-09T18:42:44.173127649Z component="gRPC Launcher" method=RequestQueries err="internal error: missing host from request context" took=1.042259ms
2020/12/09 18:42:44 http: TLS handshake error from 10.0.0.164:10314: EOF
2020/12/09 18:42:44 http: TLS handshake error from 10.0.0.164:61047: EOF
2020/12/09 18:42:45 http: TLS handshake error from 10.0.2.54:1248: EOF"

This issue seems to come from GRPC server. I searched google, but could not find an exact solution. Please see the below error image:

Therefore, could anyone help me to diagnose those error messages and fix this issue? Thank you.

Surprisingly blue at >1750px width

What version?

https://fleetdm.com as of nov 19 02:24 Central Time

What browser are you using?

Google Chrome @ latest

What did you do?

Look at the hero image @ >1750px width

What did you expect to see?

0 fault lines in the fabric of space and time

What did you see instead?

At least one such fault line, and the infinite blue beneath

I think this is probably just a positioning issue We can address this quickly by using slightly bigger image(s), e.g. up to somewhere around 2600px width.

Fleet UI Refresh

This issue describes Fleet DM’s project to “makeover” the Fleet UI. This is roughly a 2.5-week process with a focus on delivering UI design assets and implementing them via front end PRs.

Goals

This project is about dusting off the cobwebs, a sensible top, a professional haircut, and designer shoes. There will be no changes to underlying functionality or what’s possible in the UI.

Steps

• Step 1: Discovery period. Explore competing products and identify similar improvements that can be made to Fleet.
• Step 2: @edamamedesign select 2-3 pages and creates wireframes.
• Step 3: @noahtalerman and @edamamedesign review wireframes to define final outputs for high-fidelity mocks.
• Step 4: @edamamedesign creates Figma based high-fidelity mocks.
• Step 5: @noahtalerman begins implementing mocks by submitting front end PRs
• Step 6: Finish design work and front end implementation of remaining pages in tandem

Completion status

Page	Mockups	Front end
404	done	in progress (open PR)
500	done	in progress (open PR)
set up (oboarding)	done	merged (ready for review)
log in	done	in progress (open PR)
hosts	done	merged (ready for review)
query - manage	not started	merged (ready for review)
query - edit	done	merged (ready for review)
labels - new	done	merged (ready for review)
labels - edit	done	merged (ready for review)
packs - manage	not started	merged (ready for review)
packs - new	not started	merged (ready for review)
packs - edit	not started	merged (ready for review)
admin - manage users	done	merged (ready for review)
admin - app settings	done	merged (ready for review)
admin - osquery options	done	merged (ready for review)
user settings	not started	in progress (open PR)

Fleet deleted the query package, and osquery continued to query

I encountered a situation where pack was removed from Fleet UI, but the osquery client continued to execute the que

ry in pack

UI allows user to save a removed query in a pack. Save action results in 500 error.

What version of fleet are you using (fleet version --full)?

fleet - version 3.4.0-27-g27eae209
  branch: 	        master
  revision: 	27eae209fdbff5dded05570f14502d3959302d9a
  build date: 	2020-12-03T15:41:17Z
  build user: 	noahtalerman
  go version: 	go1.15.3

What operating system are you using?

macOS Big Sur Version 11.0.1

If this is a UI issue: What browser are you using?

Google Chrome Version 86.0.4240.198

If this is a performance issue: Please attach the debug archive.

What did you do?

Removed query from pack.

The query side panel on the ride isn't updated and still allows me to edit and save the removed query.

I click save and see the 500 page.

What did you expect to see?

The empty query side panel shouldn't allow me to perform actions on the now removed query.

Summary

The query side is not updated when a query is removed. This only occurs if the query was selected, as if the user was to make changes, prior to being removed. If the user removes the query without selecting the query first (not priming it for edits) then the side panel doesn't display the save action.

Add "revoke tokens" option to users UI

What version of fleet are you using (fleet version --full)?

fleet - version 3.0.0
branch: master
revision: 0058d45
build date: 2020-07-23T17:05:00Z
build user: zwass
go version: go1.14.4

What operating system are you using?

Ubuntu

What did you do?

I wanted to refresh my Bearer token so I reset my password from Fleet UI. After resetting, I got a new API key.

What did you expect to see?

I expected curl requests from the old key would fail.

What did you see instead?

Request from old, as well as new token, seems to be working.

Docker and AWS EC2 production deployment documentation

This issue describes the project of adding complete documentation for deploying Fleet in production with Docker. The documentation will include instructions for multiple deployment scenarios (e.g. helm chart for Kubernetes cluster and terraform config for AWS ECS).

Goal

Increase the ease of setting up a production environment for Fleet. The hypothesis is that flexible and informative Docker deployment documentation will encourage more potential Fleet users to experiment with a production set-up.

This issue also provides a space for questions and discussion.

Restructuring Fleet's documentation.

This issue describes a restructuring of the existing Fleet documentation.

In this issue, I've included a proposed structure for the documentation.

Please provide any suggestions or discussion in the comments!

Goal

Currently, the documentation is structured in a way that makes sense to maintainers. We'd like to restructure the docs so that they're helpful for Fleet's current and future users, visitors, and fans 🙌

Proposed structure

Below is my proposed hierarchy for the documentation. I've included the existing documentation filenames to outline the content each section will contain.

├── Using Fleet
│   ├── Fleet UI (running-queries.md, scheduling-queries.md)
│   ├── fleetctl CLI (setup-guide.md, file-format.md, file-carving.md)
│   ├── REST API (rest-endpoints.md)
│   ├── Osquery logs (working-with-osquery-logs.md)
│   ├── Monitoring Fleet (monitoring-alerting.md, performance.md)
│   ├── Security best practices (owasp-top-10.md)
│   └── Updating (updating-fleet.md)
├── Deployment
│   ├── Installation (installing-fleet.md)
│   ├── Configuration (configuring-the-fleet-binary.md, managing-osquery-configurations.md, systemd.md, single-sign-on.md)
│   ├── Adding hosts (adding-hosts-to-fleet.md)
│   └── Example deployment scenarios (fleet-on-centos.md, fleet-on-kubernetes.md, fleet-on-ubuntu.md) 
├── Contribution
│   ├── Building Fleet (building-the-code.md, development-infrastructure.md, linux.md, database-migrations.md)
│   ├── Architecture decisions (1970-01-01_template.md.md)
│   ├── Testing (testing.md)
│   └── Releasing Fleet (release.md)

Additional info about the restructuring

In this pass, most, if not all, of the copy will remain the same (we may remove duplicated copy).
In addition, the documentation will continue to live in markdown format within the main fleetdm/fleet GitHub repo.

Rename API endpoints from kolide to fleet

First we can add a deprecation warning for the old /api/v1/kolide/ endpoints.

With the 4.0.0 release we can make the breaking change of removing those in favor of /api/v1/fleet.

`fleetctl preview` creates a folder in the current working directory

Currently, fleetctl preview creates a fleet-preview/ folder in the current working directory.

Could this go somewhere else, globally? Maybe it could be created in a .tmp/ folder in the directory of the NPM package itself.

There's an argument to be made for having fleetctl preview create this directory, so you could preview more than one Fleet demo at the same time. Which is cool, but not super necessary, since it's just a demo- and the cons probably outweigh the pros.

Since Docker runs globally on the system, my expectation would be that the files related to osquery-in-a-box should exist in one place, to avoid potential confusion about which Fleet is which, and which one is being started up in Docker.

i.e. if I move or delete the fleet-preview/ folder -- and/or if I run fleetctl preview again from a different pwd

🐍 fleetctl -v
fleetctl - version 3.6.0
  branch: 	master
  revision: 	68718c183fe9288022ab84e4ca8e4439f07f7e14
  build date: 	2021-01-07T22:31:26Z
  build user: 	noahtalerman
  go version: 	go1.15.3

ui edge case: Wrong empty state and filter sidebar after canceling "Add new label"

This is a duplicate of #2326. A fix for this issue will be included in the UI Refresh #38 project.

What version of fleet are you using (fleet version --full)?

What operating system are you using?

If this is a UI issue: What browser are you using?

What did you do?

Step 1: From here, choose "Add new label"

Step 2: From here, choose "Cancel"

What did you expect to see?

What did you see instead?

In summary

Looks like the issue is that the filter stays applied but the right sidebar doesn't draw it as if it's applied, and the wrong empty state is being displayed.

Fleetctl problems on Windows

What version of fleet are you using (fleet version --full)?

3.3.0

What operating system are you using?

Windows 10

What did you do?

fleetctl login

What did you expect to see?

Successful login

What did you see instead?

PS C:\Users\zachw> fleetctl login
error verifying that config exists at ~//.fleet/config: mkdir ~\.fleet: The system cannot find the path specified.

The path looks incorrect in this case.

Manually specifying the path results in an additional error:

PS C:\Users\zachw> fleetctl login --config C:\Users\zachw\.fleet\config
error creating Fleet API client handler: loading system cert pool: crypto/x509: system root pool is not available on Windows

It's probably not a super common use case, but let's fix fleetctl to work on Windows.

Client-side sort for live query results

Goals

I, as a user, am super forgetful and will frequently execute a live query and then — while waiting for results to populate — forget the exact syntax that I used when entering the query (on the previous page).

I, as a user, have trouble easily grokking large amounts of unsorted data and would like to be able to sort Fleet’s returned query results [reverse-]alphabetically so I can quickly scroll through < dozens | hundreds | thousands > of results and get a reasonable understanding of the most common values that exist for that particular column.

Encourage fleetctl preview user to use fleetctl get enroll_secret

Currently, the top-level README in the fleetdm/fleet repo contains instructions for starting up containerized osquery agents.

Now, we instruct the user to find their enroll_secret by navigating to the "Add New Host" modal. While this step is relatively easy, I think that there's the potential for confusion. Opening the "Add New Host" modal introduces the user to an unrelated set of instructions (related to adding hosts but not helpful for adding containerized hosts).

Alternatively, the user can enter the fleetctl get enroll_secret command to obtain only their enroll_secret.

Goal

Add instructions for obtaining the user's enroll_secret by using the fleetctl get enroll_secret command. Remove the instructions that include the "Add New Host" modal.