[FEATURE REQUEST] ldap query filter supports regualr expression

[ctaoist]

[FEATURE REQUEST] ldap query filter supports regualr expression

[nitnelave]

I'm surprised by this behavior. The wildcard should be translated into a substring filter, which is supported.

Can you add the LLDAP verbose logs of receiving/replying to the query?

[ctaoist]

I'm surprised by this behavior. The wildcard should be translated into a substring filter, which is supported.

Can you add the LLDAP verbose logs of receiving/replying to the query?

I send a query (mail=*@example.com) when adding a breakpoint at

lldap/server/src/infra/ldap_handler.rs

Line 787 in 2ca5e9e

LdapOp::SearchRequest(request) => self

and the filter is:

As we seen, that the * was discard.

[nitnelave]

That actually seems perfectly fine: it's a substring filter that means "ends with @example.com"

Do you have the verbose LLDAP logs for the query?

[ctaoist]

That actually seems perfectly fine: it's a substring filter that means "ends with @example.com"

Do you have the verbose LLDAP logs for the query?

It finally returns the empty search result even if there has many records ending with @example.com

When I change the query to ([email protected]), the value of the filter at the breakpoint location would be [email protected], and It can return right search result.

verbose:

Loading configuration from /root/apps/openldap/lldap_config.toml
Configuration: Configuration {
    ldap_host: "0.0.0.0",
    ldap_port: 3890,
    http_host: "0.0.0.0",
    http_port: 17170,
    jwt_secret: ***SECRET***,
    ldap_base_dn: "dc=exmaple,dc=com",
    ldap_user_dn: UserId(
        "admin",
    ),
    ldap_user_email: "[email protected]",
    ldap_user_pass: ***SECRET***,
    database_url: "sqlite:////root/openldap/users.db?mode=rwc",
    ignored_user_attributes: [],
    ignored_group_attributes: [],
    verbose: true,
    key_file: "/data/private_key",
    key_seed: Some(
        ***SECRET***,
    ),
    smtp_options: MailOptions {
        enable_password_reset: false,
        from: Some(
            Mailbox {
                name: Some(
                    "LLDAP Admin",
                ),
                email: Address {
                    serialized: "[email protected]",
                    at_start: 7,
                },
            },
        ),
        reply_to: Some(
            Mailbox {
                name: Some(
                    "Do not reply",
                ),
                email: Address {
                    serialized: "noreply@localhost",
                    at_start: 7,
                },
            },
        ),
        server: "10.0.0.2",
        port: 465,
        user: "admin",
        password: ***SECRET***,
        smtp_encryption: Tls,
        tls_required: None,
    },
    ldaps_options: LdapsOptions {
        enabled: false,
        port: 6360,
        cert_file: "cert.pem",
        key_file: "key.pem",
    },
    http_url: Url {
        scheme: "http",
        cannot_be_a_base: false,
        username: "",
        password: None,
        host: Some(
            Domain(
                "localhost",
            ),
        ),
        port: None,
        path: "/",
        query: None,
        fragment: None,
    },
    server_setup: None,
}
WARNING: A key_seed was given, we will ignore the server_key and generate one from the seed!
2023-11-18T14:01:05.889546702+00:00  INFO     set_up_server [ 22.1ms | 49.22% / 100.00% ]
2023-11-18T14:01:05.889878912+00:00  INFO     ┝━ ｉ [info]: Starting LLDAP version 0.5.1-alpha
2023-11-18T14:01:05.894297851+00:00  DEBUG    ┝━ get_schema_version [ 1.22ms | 5.51% ]
2023-11-18T14:01:05.897385323+00:00  DEBUG    │  ┕━ 🐛 [debug]:  | return: Some(SchemaVersion(5))
2023-11-18T14:01:05.897832813+00:00  DEBUG    ┝━ list_groups [ 3.78ms | 17.08% ] filters: Some(DisplayName("lldap_admin"))
2023-11-18T14:01:05.904769845+00:00  DEBUG    │  ┕━ 🐛 [debug]:  | return: [Group { id: GroupId(1), display_name: "lldap_admin", creation_date: 2023-11-15T05:08:19.259454380, uuid: Uuid("ea0a5a52-c37d-3832-aaeb-53959cfeebca"), users: [UserId("admin")], attributes: [] }]
2023-11-18T14:01:05.904908532+00:00  DEBUG    ┝━ list_groups [ 1.63ms | 7.39% ] filters: Some(DisplayName("lldap_password_manager"))
2023-11-18T14:01:05.907366246+00:00  DEBUG    │  ┕━ 🐛 [debug]:  | return: [Group { id: GroupId(2), display_name: "lldap_password_manager", creation_date: 2023-11-15T05:08:19.296152273, uuid: Uuid("3c86ade7-3f7b-3ef4-8c06-c677b8cfe2ae"), users: [], attributes: [] }]
2023-11-18T14:01:05.907469770+00:00  DEBUG    ┝━ list_groups [ 1.60ms | 7.22% ] filters: Some(DisplayName("lldap_strict_readonly"))
2023-11-18T14:01:05.909888799+00:00  DEBUG    │  ┕━ 🐛 [debug]:  | return: [Group { id: GroupId(3), display_name: "lldap_strict_readonly", creation_date: 2023-11-15T05:08:19.328653117, uuid: Uuid("0349e9c4-1e5f-3583-a2ae-892777f09e72"), users: [UserId("readonly")], attributes: [] }]
2023-11-18T14:01:05.910044493+00:00  DEBUG    ┝━ list_users [ 2.36ms | 10.67% ] filters: Some(MemberOf("lldap_admin")) | _get_groups: false
2023-11-18T14:01:05.914345265+00:00  DEBUG    │  ┕━ 🐛 [debug]:  | return: [UserAndGroups { user: User { user_id: UserId("admin"), email: "[email protected]", display_name: Some("Administrator"), creation_date: 2023-11-15T05:08:19.359780369, uuid: Uuid("3c4f7253-a57a-36c9-bdd5-20d90f6453f3"), attributes: [AttributeValue { name: "userpassword", value: Serialized("$6$jps1xxxxxxxxxxxx") }] }, groups: Some([GroupDetails { group_id: GroupId(1), display_name: "lldap_admin", creation_date: 2023-11-15T05:08:19.259454380, uuid: Uuid("ea0a5a52-c37d-3832-aaeb-53959cfeebca"), attributes: [] }, GroupDetails { group_id: GroupId(4), display_name: "mail", creation_date: 2023-11-15T07:06:55.983026925, uuid: Uuid("61926071-d327-3557-894b-79dcb7c23630"), attributes: [] }]) }]
2023-11-18T14:01:05.918014440+00:00  INFO     ┝━ ｉ [info]: Starting the LDAP server on port 3890
2023-11-18T14:01:05.921473341+00:00  DEBUG    ┝━ get_jwt_blacklist [ 646µs | 2.92% ]
2023-11-18T14:01:05.922799330+00:00  INFO     ┕━ ｉ [info]: Starting the API/web server on port 17170
2023-11-18T14:01:05.923939011+00:00  INFO     ｉ [info]: starting 1 workers
2023-11-18T14:01:05.924172835+00:00  INFO     ｉ [info]: Actix runtime found; starting in Actix runtime
2023-11-18T14:01:05.927034045+00:00  ERROR    🚨 [error]: Specified path is not a directory: "./app/pkg" | log.target: "actix_files::files" | log.module_path: "actix_files::files" | log.file: "/root/.cargo/registry/src/rsproxy.cn-0dccff568467c15b/actix-files-0.6.2/src/files.rs" | log.line: 104
2023-11-18T14:01:05.930004480+00:00  INFO     ｉ [info]: DB Cleanup Cron started

There was no new debug output when I send search queries.

[nitnelave]

If there's no new debug output, it's because no query reached the server: we log something for every query. Is there a cache involved, maybe?

I'd really need to see the logs of the query with the substring filter

[ctaoist]

If there's no new debug output, it's because no query reached the server: we log something for every query. Is there a cache involved, maybe?

I'd really need to see the logs of the query with the substring filter

There would be debug output until shutting down the lldap progress by using Ctrl-c.

verbose:

2023-11-19T02:55:03.471274312+00:00  INFO     ｉ [info]: SIGINT received; starting forced shutdown
2023-11-19T02:55:03.471796566+00:00  INFO     ｉ [info]: force shutdown worker, closing 1 connections
2023-11-19T02:55:03.471756942+00:00  DEBUG    🐛 [debug]: paused accepting connections on 0.0.0.0:3890
2023-11-19T02:55:03.472051596+00:00  DEBUG    🐛 [debug]: paused accepting connections on 0.0.0.0:17170
2023-11-19T02:55:03.472132993+00:00  INFO     ｉ [info]: accept thread stopped
2023-11-19T02:54:56.612851480+00:00  INFO     LDAP session [ 1.44s | 0.12% / 100.00% ]
2023-11-19T02:54:56.616896412+00:00  INFO     ┝━ LDAP request [ 1.44s | 0.05% / 99.59% ]
example,dc=com", cred: LdapBindCred::Simple }), ctrl: [] }
example,dc=com
2023-11-19T02:54:56.617357233+00:00  DEBUG    │  │  ┝━ bind [ 1.43s | 0.02% / 99.40% ]
2023-11-19T02:54:56.617426196+00:00  DEBUG    │  │  │  ┝━ get_password_file_for_user [ 1.24ms | 0.09% ] user_id: UserId("admin")
2023-11-19T02:54:56.619657472+00:00  DEBUG    │  │  │  ┕━ passwords_match [ 1.43s | 99.29% ] username: admin
2023-11-19T02:54:58.051197369+00:00  DEBUG    │  │  ┝━ get_user_groups [ 1.54ms | 0.11% ] user_id: "admin"
2023-11-19T02:54:58.055223269+00:00  DEBUG    │  │  │  ┕━ 🐛 [debug]:  | return: {GroupDetails { group_id: GroupId(1), display_name: "lldap_admin", creation_date: 2023-11-15T05:08:19.259454380, uuid: Uuid("ea0a5a52-c37d-3832-aaeb-53959cfeebca"), attributes: [] }, GroupDetails { group_id: GroupId(4), display_name: "mail", creation_date: 2023-11-15T07:06:55.983026925, uuid: Uuid("61926071-d327-3557-894b-79dcb7c23630"), attributes: [] }}
2023-11-19T02:54:58.055292526+00:00  DEBUG    │  │  ┕━ 🐛 [debug]: Success!
2023-11-19T02:54:58.055433651+00:00  DEBUG    │  ┕━ 🐛 [debug]:  | response: BindResponse(LdapBindResponse { res: LdapResult { code: Success, matcheddn: "", message: "", referral: [] }, saslcreds: None })
---------------------------------------------------------------------------------------------------------
-------------------------------------- query: `(mail=*@example.com)` ------------------------------------
---------------------------------------------------------------------------------------------------------
2023-11-19T02:55:01.095867703+00:00  INFO     ┕━ LDAP request [ 4.27ms | 0.04% / 0.30% ]
example,dc=com", scope: Subtree, aliases: Always, sizelimit: 0, timelimit: 0, typesonly: false, filter: Substring("mail", LdapSubstringFilter { initial: None, any: [], final_: Some("@example.com") }), attrs: ["uid"] }), ctrl: [] }
2023-11-19T02:55:01.096077717+00:00  DEBUG       ┝━ do_search [ 3.67ms | 0.03% / 0.25% ]
example,dc=com" | scope: Global
2023-11-19T02:55:01.096295187+00:00  DEBUG       │  ┝━ get_user_list [ 3.23ms | 0.02% / 0.22% ]
2023-11-19T02:55:01.096369614+00:00  DEBUG       │  │  ┝━ 🐛 [debug]:  | filters: SubString(Email, SubStringFilter { initial: None, any: [], final_: Some("@example.com") })
2023-11-19T02:55:01.096422834+00:00  DEBUG       │  │  ┕━ list_users [ 3.01ms | 0.21% ] filters: Some(SubString(Email, SubStringFilter { initial: None, any: [], final_: Some("@example.com") })) | _get_groups: false
2023-11-19T02:55:01.102490529+00:00  DEBUG       │  │     ┕━ 🐛 [debug]:  | return: [UserAndGroups { user: User { user_id: UserId("admin"), email: "[email protected]", display_name: Some("Administrator"), creation_date: 2023-11-15T05:08:19.359780369, uuid: Uuid("3c4f7253-a57a-36c9-bdd5-20d90f6453f3"), attributes: [AttributeValue { name: "userpassword", value: Serialized("$6$jpxx") }] }, groups: Some([GroupDetails { group_id: GroupId(1), display_name: "lldap_admin", creation_date: 2023-11-15T05:08:19.259454380, uuid: Uuid("ea0a5a52-c37d-3832-aaeb-53959cfeebca"), attributes: [] }, GroupDetails { group_id: GroupId(4), display_name: "mail", creation_date: 2023-11-15T07:06:55.983026925, uuid: Uuid("61926071-d327-3557-894b-79dcb7c23630"), attributes: [] }]) }, UserAndGroups { user: User { user_id: UserId("ctaoist"), email: "[email protected]", display_name: Some("ctaoist"), creation_date: 2023-11-15T05:12:09.139400973, uuid: Uuid("93c41d81-a7fb-31d3-9e2e-1365077b49a3"), attributes: [] }, groups: Some([GroupDetails { group_id: GroupId(5), display_name: "gitea", creation_date: 2023-11-15T07:07:18.848095080, uuid: Uuid("363470da-b221-370e-b7dd-f343ad9d7288"), attributes: [] }, GroupDetails { group_id: GroupId(4), display_name: "mail", creation_date: 2023-11-15T07:06:55.983026925, uuid: Uuid("61926071-d327-3557-894b-79dcb7c23630"), attributes: [] }]) }]
2023-11-19T02:55:01.102646176+00:00  DEBUG       │  ┕━ get_groups_list [ 33.6µs | 0.00% ]
2023-11-19T02:55:01.102794393+00:00  DEBUG       ┕━ 🐛 [debug]:  | response: SearchResultDone(LdapResult { code: UnwillingToPerform, matcheddn: "", message: "Unsupported group attribute for substring filter: \"mail\"", referral: [] })
2023-11-19T02:55:03.775054411+00:00  INFO     ｉ [info]: DB Cleanup stopped
2023-11-19T02:55:03.775594692+00:00  INFO     ｉ [info]: End.

The error message is Unsupported group attribute for substring filter: \"mail\" at get_groups_list function.

I have already set ignored_group_attributes in lldap_config.toml:

## Ignored attributes.
## Some services will request attributes that are not present in LLDAP. When it
## is the case, LLDAP will warn about the attribute being unknown. If you want
## to ignore the attribute and the service works without, you can add it to this
## list to silence the warning.
#ignored_user_attributes = [ "sAMAccountName" ]
ignored_group_attributes = [ "mail", "userPrincipalName" ]

I use the python3 ldap3 as my ldap client for test:

import ldap3
from ldap3 import Server, Connection

conn = Connection(Server('localhost',port=3890, get_info='all'), 'uid=admin,ou=people,dc=example,dc=com', 'password')
conn.bind()

conn.search('dc=example,dc=com', f'(mail=*@ctaoist.cn)', attributes=['uid'])
conn.entries

If I change base_dn from dc=example,dc=com to ou=people,dc=example,dc=com, it works fine. I am very sorry about that I am a newer with ldap.

BTW, the time in debug output is utc time insdead of local time.

[nitnelave]

Oh, I see what's happening: the query works fine and finds the right users, but since the base DN is broad it also looks for groups and runs into an error (substring for email is not supported for groups). That aborts the whole query. It should be easy to fix. And yes, as you found out, changing the base DN, or adding a (objectClass=person) to the query would solve the problem.

…

On Sun, 19 Nov 2023, 05:37 ctaoist, ***@***.***> wrote: If there's no new debug output, it's because no query reached the server: we log something for every query. Is there a cache involved, maybe? I'd really need to see the logs of the query with the substring filter There would be debug output until shutting down the lldap progress by using Ctrl-c. verbose: 2023-11-19T02:55:03.471274312+00:00 INFO ｉ [info]: SIGINT received; starting forced shutdown 2023-11-19T02:55:03.471796566+00:00 INFO ｉ [info]: force shutdown worker, closing 1 connections 2023-11-19T02:55:03.471756942+00:00 DEBUG 🐛 [debug]: paused accepting connections on 0.0.0.0:3890 2023-11-19T02:55:03.472051596+00:00 DEBUG 🐛 [debug]: paused accepting connections on 0.0.0.0:17170 2023-11-19T02:55:03.472132993+00:00 INFO ｉ [info]: accept thread stopped 2023-11-19T02:54:56.612851480+00:00 INFO LDAP session [ 1.44s | 0.12% / 100.00% ] 2023-11-19T02:54:56.616896412+00:00 INFO ┝━ LDAP request [ 1.44s | 0.05% / 99.59% ] example,dc=com", cred: LdapBindCred::Simple }), ctrl: [] } example,dc=com 2023-11-19T02:54:56.617357233+00:00 DEBUG │ │ ┝━ bind [ 1.43s | 0.02% / 99.40% ] 2023-11-19T02:54:56.617426196+00:00 DEBUG │ │ │ ┝━ get_password_file_for_user [ 1.24ms | 0.09% ] user_id: UserId("admin") 2023-11-19T02:54:56.619657472+00:00 DEBUG │ │ │ ┕━ passwords_match [ 1.43s | 99.29% ] username: admin 2023-11-19T02:54:58.051197369+00:00 DEBUG │ │ ┝━ get_user_groups [ 1.54ms | 0.11% ] user_id: "admin" 2023-11-19T02:54:58.055223269+00:00 DEBUG │ │ │ ┕━ 🐛 [debug]: | return: {GroupDetails { group_id: GroupId(1), display_name: "lldap_admin", creation_date: 2023-11-15T05:08:19.259454380, uuid: Uuid("ea0a5a52-c37d-3832-aaeb-53959cfeebca"), attributes: [] }, GroupDetails { group_id: GroupId(4), display_name: "mail", creation_date: 2023-11-15T07:06:55.983026925, uuid: Uuid("61926071-d327-3557-894b-79dcb7c23630"), attributes: [] }} 2023-11-19T02:54:58.055292526+00:00 DEBUG │ │ ┕━ 🐛 [debug]: Success! 2023-11-19T02:54:58.055433651+00:00 DEBUG │ ┕━ 🐛 [debug]: | response: BindResponse(LdapBindResponse { res: LdapResult { code: Success, matcheddn: "", message: "", referral: [] }, saslcreds: None }) --------------------------------------------------------------------------------------------------------- -------------------------------------- query: ***@***.***)` ------------------------------------ --------------------------------------------------------------------------------------------------------- 2023-11-19T02:55:01.095867703+00:00 INFO ┕━ LDAP request [ 4.27ms | 0.04% / 0.30% ] example,dc=com", scope: Subtree, aliases: Always, sizelimit: 0, timelimit: 0, typesonly: false, filter: Substring("mail", LdapSubstringFilter { initial: None, any: [], final_: ***@***.***") }), attrs: ["uid"] }), ctrl: [] } 2023-11-19T02:55:01.096077717+00:00 DEBUG ┝━ do_search [ 3.67ms | 0.03% / 0.25% ] example,dc=com" | scope: Global 2023-11-19T02:55:01.096295187+00:00 DEBUG │ ┝━ get_user_list [ 3.23ms | 0.02% / 0.22% ] 2023-11-19T02:55:01.096369614+00:00 DEBUG │ │ ┝━ 🐛 [debug]: | filters: SubString(Email, SubStringFilter { initial: None, any: [], final_: ***@***.***") }) 2023-11-19T02:55:01.096422834+00:00 DEBUG │ │ ┕━ list_users [ 3.01ms | 0.21% ] filters: Some(SubString(Email, SubStringFilter { initial: None, any: [], final_: ***@***.***") })) | _get_groups: false 2023-11-19T02:55:01.102490529+00:00 DEBUG │ │ ┕━ 🐛 [debug]: | return: [UserAndGroups { user: User { user_id: UserId("admin"), email: ***@***.***", display_name: Some("Administrator"), creation_date: 2023-11-15T05:08:19.359780369, uuid: Uuid("3c4f7253-a57a-36c9-bdd5-20d90f6453f3"), attributes: [AttributeValue { name: "userpassword", value: Serialized("$6$jpxx") }] }, groups: Some([GroupDetails { group_id: GroupId(1), display_name: "lldap_admin", creation_date: 2023-11-15T05:08:19.259454380, uuid: Uuid("ea0a5a52-c37d-3832-aaeb-53959cfeebca"), attributes: [] }, GroupDetails { group_id: GroupId(4), display_name: "mail", creation_date: 2023-11-15T07:06:55.983026925, uuid: Uuid("61926071-d327-3557-894b-79dcb7c23630"), attributes: [] }]) }, UserAndGroups { user: User { user_id: UserId("ctaoist"), email: ***@***.***", display_name: Some("ctaoist"), creation_date: 2023-11-15T05:12:09.139400973, uuid: Uuid("93c41d81-a7fb-31d3-9e2e-1365077b49a3"), attributes: [] }, groups: Some([GroupDetails { group_id: GroupId(5), display_name: "gitea", creation_date: 2023-11-15T07:07:18.848095080, uuid: Uuid("363470da-b221-370e-b7dd-f343ad9d7288"), attributes: [] }, GroupDetails { group_id: GroupId(4), display_name: "mail", creation_date: 2023-11-15T07:06:55.983026925, uuid: Uuid("61926071-d327-3557-894b-79dcb7c23630"), attributes: [] }]) }] 2023-11-19T02:55:01.102646176+00:00 DEBUG │ ┕━ get_groups_list [ 33.6µs | 0.00% ] 2023-11-19T02:55:01.102794393+00:00 DEBUG ┕━ 🐛 [debug]: | response: SearchResultDone(LdapResult { code: UnwillingToPerform, matcheddn: "", message: "Unsupported group attribute for substring filter: \"mail\"", referral: [] }) 2023-11-19T02:55:03.775054411+00:00 INFO ｉ [info]: DB Cleanup stopped 2023-11-19T02:55:03.775594692+00:00 INFO ｉ [info]: End. The error message is Unsupported group attribute for substring filter: \"mail\" at get_groups_list function. I have already set ignored_group_attributes in lldap_config.toml: ## Ignored attributes. ## Some services will request attributes that are not present in LLDAP. When it ## is the case, LLDAP will warn about the attribute being unknown. If you want ## to ignore the attribute and the service works without, you can add it to this ## list to silence the warning. #ignored_user_attributes = [ "sAMAccountName" ] ignored_group_attributes = [ "mail", "userPrincipalName" ] I use the python3 ldap3 as my ldap client for test: import ldap3from ldap3 import Server, Connection conn = Connection(Server('localhost',port=3890, get_info='all'), 'uid=admin,ou=people,dc=example,dc=com', 'password')conn.bind() conn.search('dc=example,dc=com', ***@***.***)', attributes=['uid'])conn.entries If I change base_dn from dc=example,dc=com to ou=people,dc=example,dc=com, it works fine. I am very sorry about that I am a newer with ldap. BTW, the time in debug output is utc time insdead of local time. — Reply to this email directly, view it on GitHub <#738 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGCPWJVWF3NXTJ2TAJ2GGLYFGEIDAVCNFSM6AAAAAA7Q3KXUOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJXG42DGMRVGI> . You are receiving this because you commented.Message ID: ***@***.***>

[ctaoist]

Thanks a lot. I closed this issue.

[nitnelave]

I'll reopen since it shouldn't return an error.

[pixelrazor]

What's the intended behavior here? both user and group search is done, and only one fails, log that failure as warning then return the successful results?

[nitnelave]

Yes, I'm thinking more generally that an attribute failure for either users or groups, when we search for both, should not be an error.

…

On Mon, 20 Nov 2023, 15:46 Austin Alvarado, ***@***.***> wrote: What's the intended behavior here? both user and group search is done, and only one fails, log that failure as warning then return the successful results? — Reply to this email directly, view it on GitHub <#738 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGCPWPHLZAJN6SODW5ASCDYFNUN5AVCNFSM6AAAAAA7Q3KXUOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJZGIYDIMJXHA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>