ExceptFromCloudflare seems to have no effect

[Ivan-Kouznetsov]

ExceptFromCloudflare seems to have no effect

[dustinmoris]

Hi, thanks for raising the issue. I'll look into this tomorrow and let you know an ETA!

[dustinmoris]

Ok, so the X-Forwarded-For header is used so that a web application can be notified about the origin IP address in a chain of server infrastructure.

In your example it may look a bit like this:

User on web browser ---- User IP ---> Cloudflare
Cloudflare ---- Cloudflare IP ----> Kubernetes
Kubernetes Load Balancer ---> LB IP ----> nginx container
nginx container -----> private container IP ----> web app container
web app container sees the last IP, but doesn't know about any of the other IP addresses.

X-Forwarded-For helps to overcome this issue. It is a HTTP header which intermediate infrastructure can use to store the previous request's IP address in the header, so the end application can look at the X-Forwarded-For header to know where the request really came from.

However, like in your example, there is more than just one server inbetween you and the client's web browser. There is nginx, cloudflare, a load balancer, etc., this is why X-Forwarded-For doesn't just allow to store a single IP address, but a list of IP addresses, so the end application can see the entire chain from where the request came from.

This is the first point to know, because if you write some custom rules in your nginx to set the X-Forwarded-For header with a new value, then you basically erradicate the previous history which would include Cloudflare's IP and the end application will not be able to evaluate if it came from Cloudflar eor not. So you should only append, not overwrite this value.

The other thing to note is that the X-Forwarded-For header is something which can be set by the user to begin with, so you should be vary careful in trusting this header and shouldn't trust an endless chain of IP addresses. You should only trust the number of servers which you know are inbetween you and cloudflare based on your deployment setup.

I hope this little piece of information will help you to orderly configure the middleware in ASP.NET Core. If it's not correctly configured then you will just not see Cloudflare's IP address in the chain as part of X-Forwarded-For.

[Trojaner]

I have the same issue.
Do you use .NET Core 3 with the new Routing API? Maybe it is related to that

[Ivan-Kouznetsov]

I think it may be caused by Cloudflare's lists of IP addresses simply being out of date, to check for that just display the IP address when a request is rejected. When I did that and tried accessing the site via a domain that uses Cloudflare I saw IP addresses that were not on the Cloudflare lists. I contacted them about it but as of yet received no reply.

[dustinmoris]

Hi, I've just tested this and it worked for me as expected. I've upgraded the sample app on my machine to .NET Core 3 as well and it just works.

May I ask how you host your application? Because I suspect that you are running into an issue where there is other infrastructure between your app and cloudflare (e.g. when you run your app as a Docker container in a cluster behind a load balancer) and you don't forward Cloudflare's IP address to your ASP.NET Core app via the X-Forwarded-For http header.

Please have a look at this in the documentation: https://github.com/dustinmoris/Firewall#x-forwarded-for-http-header

When I did that and tried accessing the site via a domain that uses Cloudflare I saw IP addresses that were not on the Cloudflare lists.

Please note that the addresses which you see in the lists are CIDRs, which are address spaces and not actual IP addresses, so what you will see in your console output will most certainly look different that what you see on that list. This is part of the reason why I had to write this library, to do the actual work of checking an IP address to be within a given CIDR range :).

[Ivan-Kouznetsov]

May I ask how you host your application?

It's an ASP.NET Core 2.1 Razor Pages application hosted via a shared hosting provider on IIS on Windows

Please note that the addresses which you see in the lists are CIDRs, which are address spaces and not actual IP addresses, so what you will see in your console output will most certainly look different that what you see on that list. This is part of the reason why I had to write this library, to do the actual work of checking an IP address to be within a given CIDR range :).

Right, I mean that it was my impression that the IP was not in the ranges provided, for example, I saw a connection from 192.168.70.57 which does not seem like it would be in the ip4 ranges they provided unless I am misunderstanding something.

[dustinmoris]

Right, I mean that it was my impression that the IP was not in the ranges provided, for example, I saw a connection from 192.168.70.57 which does not seem like it would be in the ip4 ranges they provided unless I am misunderstanding something.

Any IP address which begins with 192.168.x.x is a so called private IP address, which means it didn't come from Cloudflare but your hosting provider's private network. Private IP addresses are reserved for local networks, you can also read about it here.

Now seeing that you are getting a request from 192.168.70.57 it is 100% the case that you have some sort of intermediate infrastructure (maybe a load balancer?) which makes the actual request to your ASP.NET Core application. If you configure your forwarded for headers then it should (hopefully) fix your issue.

Start with ForwardLimit = 1 assuming that you have only one intermediary between your app and Cloudflare and only increase that number when the issue still persists. Once you set this number to more than 1, then you have to be careful and possibly configure the trusted IP addresses in the chain, but I don't want to confuse you with this before necessary :)

[Ivan-Kouznetsov]

Thanks!

[dustinmoris]

Please let me know here how you get on with the issue, because I'm sure that others will potentially also experience the same or similar problem and then they will probably stuble across this thread which could help them! Thank you

[Trojaner]

May I ask how you host your application? Because I suspect that you are running into an issue where there is other infrastructure between your app and cloudflare (e.g. when you run your app as a Docker container in a cluster behind a load balancer) and you don't forward Cloudflare's IP address to your ASP.NET Core app via the X-Forwarded-For http header.

This is exactly the infrastructure I am using (docker in combination with Cloudflare).
I am using the Forwarded Headers middleware like this:

            var forwardOptions = new ForwardedHeadersOptions
            {
                ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
            };
            forwardOptions.KnownNetworks.Clear(); // by default it adds loopback
            forwardOptions.KnownProxies.Clear(); // by default it adds loopback

            app.UseForwardedHeaders(forwardOptions);
            var rules =
                FirewallRulesEngine
                    .DenyAllAccess()
                    .ExceptFromCloudflare()
                    .ExceptFromLocalhost();

            app.UseFirewall(rules);

I am setting the X-Forwarded-For header in nginx like this:

proxy_set_header X-Forwarded-For $remote_addr;

In my logs with RemoteIpAddress the correct IP address of the user shows up, but when enabling UseFirewall after Forwarded Headers middleware it does not work and all requests get denied

[Trojaner]

Thank you, I will try these out and provide feedback if it still does not work.

[Trojaner]

After double checking everything and fixing the issues you have mentioned it still does not work.

The Forwarded Header Middleware sets the user IP correctly.

ExceptFromCloudflare only checks against the RemoteIpAddress property (which is the user IP address) but not the other addresses of the X-Forwarded-For header (which contains Cloudflare's IP address)

IPAddressRule and ExceptFromCloudflare should have an option to check against all IP addresses in the X-Forwarded-For header. Of course, like you have mentioned earlier, this requires a safe configuration.

Otherwise, ExceptFromCloudflare can only work if the RemoteIpAddress property is not touched (e.g. by UseForwardedHeaders()) and only contains Cloudflare's IP. This is also often not the case since most services also run behind a Load Balancer, in a container, etc.

[jollynv]

I am facing the same issue where my app is running in a docker container. ExceptFromCloudflare() restricts even the incoming traffic through Cloudflare. Adding X-Forwarded-For doesn't fix the issue.
Any plans/movement on the above issue's resolution?

[dustinmoris]

Cloudflare publishes all it's IP ranges via two lists:

• https://www.cloudflare.com/ips-v4
• https://www.cloudflare.com/ips-v6

These IP ranges get added as an exception to the rule and should work. If it doesn't work for you then it is most likely because your incoming request comes from an intermediate proxy server. The best way to debug this is to log the IP address from your requests and then check if it belongs to one of the above listed IP ranges. My guess is that it's not going to be and in fact I even bet that you'll find that the remote IP address is probably an internal address from your network where some load balancer or other proxy has forwarded the original request from Cloudflare. You must figure out how many proxies are between you and Cloudflare and then correctly configure your Forwarded For middleware otherwise ASP.NET Core will not propagate the correct IP address as the remote address and then the Firewall will not know that the request came through Cloudflare.