Imperva Cyber Community

communities_1.jpg
 View Only
Expand all | Collapse all

ABP and F5 Connector

  • 1.  ABP and F5 Connector

    Posted 09-10-2020 17:17

    Hello everyone,

    I have been using Distil Networks ( on-prem deployment ) for some time but after its acquisition with Imperva, I am on a mission to migrate it from on-prem design to Imperva Cloud.

    I took the F5 Connector path for this journey because our applications are tightly coupled with F5 Load balancers.
    F5 Connector

    I was wondering if someone here in this community has crossed this bridge and would like to share his experience or may help me out.

    Currently, I have gone through the document and completed the following tasks :

    • Configure the javascript and tcl files
    • Create the F5 plugin
    • Upload the plugin to the workspace
    • Create a new pool for the analysis request
    • Enable protection on a virtual server

    It appears I am not receiving "cookies" from Imperva Cloud.

    Basically, I want to know how to test the setup and what to look for in the headers that will confirm that it is working or if not then where it is breaking.

    I highly appreciate in advance for any insight, tips or hints that will help me to land this plane sooner and safely.

    Thank you

    Muhammad Shoaib


    #AdvancedBotProtection

    ------------------------------
    Muhammad Shoaib
    Paciolan Inc.
    CA
    ------------------------------


  • 2.  RE: ABP and F5 Connector

    Posted 09-11-2020 13:21
    Edited by Christopher Detzel 09-11-2020 13:52
    Hi Muhammad,

    To start, be sure that the challenge tag (e.g. <script type="text/javascript" src="<challenge-path-value>" async></script>) is included on the page you are testing on and the challenge path value you place on the page matches the challenge path configured in the iRule.

    The easiest way to test if the requests are making it to us or not is to create a policy that blocks based on a specific header and then sending a request to your site with that header. If you see our block page, you'll know that the request is making it to us.

    To create the policy:
    1. Log into the Imperva Bot Management Console
    2. Click on Connector
    3. Make sure the site you want to check shows up in the Sites list (if the site is not in the list, you will need to create it and probably need to update the F5 connector configuration with the credentials that are provided)
    4. Click on Policies
    5. Click on the default policy for the site 
    6. Under the block section, click on Add Condition
    7. Click on New Conditions and then click on Create next to Header
    8. Give the condition a name such as connector block test
    9. Fill out the header name with the header key you want to use, e.g. block-test
    10. Fill out the header pattern with the header value you want to use, e.g. block
    11. Click Save
    12. Click Publish Configuration

    To test the policy you can use an extension for the browser of your choice (such as Modify Header Value (HTTP Headers) for Chrome or Firefox) to insert the header and value that you created in the policy. Alternatively, you could use curl to test by including the -H option and providing the header details from the policy you created, e.g. curl -H 'block-test: block' http://example.com. However you send the request, you should see the Imperva block page.

    If you don't see the block page, the first place to start troubleshooting is the LTM logs on the F5 load balancer (located at /var/log/ltm). You may see an error like the following: 
    Sep 11 09:30:45 <hostname> err tmm1[15150]: Rule /Common/imperva-f5/imperva-https <HTTP_REQUEST>: Client - <ip address>, Imperva failure :remoteAnalyze: Error: <error message>

    The error message provided should give you a clue about why traffic to the analysis host is not working. Some possibilities are incorrect analysis host address, incorrect api keys, the domain is not created in the portal, or inability to route to the analysis host (this list is not exhaustive but just a few examples).

    Take corrective action based on the error message you see and try again. If you don't see any log entries related to the connector iRule, it's likely that the rule is either not applied to the virtual server or the rule was not implemented correctly. Double check the implementation steps.

    If you are unable to get the integration working on your own, you are always welcome to open a support ticket and we'll do our best to get you sorted.

    Good Luck!

    Derrick Lowder

    ------------------------------
    Derrick Lowder
    Tech Lead - ABP Operations
    Imperva
    ------------------------------



  • 3.  RE: ABP and F5 Connector

    Posted 09-14-2020 15:34
    Edited by Muhammad Shoaib 09-14-2020 15:36

    Derrick ... first of all, thank you for the detailed response with step by step instructions.

    I have gone through the steps below and had partial success:

    1. Log into the Imperva Bot Management Console (https://bots.imperva.com/) - I don't have an account on this instead I logged in to (https://my.imperva.com) and then clicked on the ABP icon in the left Nav-bar and picked up your steps from #4 - Policies
    2. Policies > Default Policies > Block Section > Insert Condition >
      Header Name: X-Block-Imperva
      Header Pattern: abcxyz123
    3. Save & Published
    4. It worked and I see a page "Pardon Out Interruption"
      wooohooo !!!

      I tried the Chrome Extension and it works great inserting test header to my requests.


    I tried to Move the above condition under captcha then I see the following error under BigIP logs ( /var/log/ltm ).

    I was expecting to see a captcha page. Do I need to configure anything extra for Captcha to work? either on BigIP or on Imperva ABP portal?

    Please advise.

    Sep 14 12:09:55 LTM2000 err tmm1[10205]: Rule /Common/imperva/imperva <HTTP_REQUEST>: Client - 68.99.182.31, Imperva failure :proxyInterstitialPage: TypeError: body used already for: https://<part removed xxxxx>.impervaabp.net/v6/captcha/<part removed xxxxx>/my-challenge-path

    Sep 14 12:09:56 LTM2000 err tmm[10205]: Rule /Common/imperva/imperva <HTTP_REQUEST>: Client - 68.99.182.31, Imperva failure :proxyInterstitialPage: TypeError: body used already for: https://<part removed xxxxx>.impervaabp.net/v6/captcha/<part removed xxxxx>/my-challenge-path

    ​Thank you for your help

    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------



  • 4.  RE: ABP and F5 Connector

    Posted 09-15-2020 11:55
    Hey Muhammad,

    You may need to enable captcha in the portal. 

    1. Log into https://my.imperva.com
    2. Click on Advanced Bot Protection in the sidebar
    3. Click Launch Advanced Bot Protection
    4. Click Settings
    5. Click on the Website Group for the domain you wish to modify
    6. Under Website Configuration, click on the domain you wish to modify
    7. Click Show advanced settings
    8. Scroll to the bottom to find Captcha Settings
    9. Change the selection from None to the captcha provider of your choice (Note: custom geetest and recaptcha will require you to provide API keys. you can test with geetest without providing a key).
    10. Click Save
    11. Click Publish Configuration

    Try that and let me know if captcha is working. If it does not work, let me know if the error message you see in the log changes. 




    ------------------------------
    Derrick Lowder
    Tech Lead - ABP Operations
    Imperva, Inc
    ------------------------------



  • 5.  RE: ABP and F5 Connector

    Posted 09-15-2020 12:28
    Hi Derrick,

    I don't see an option "advance settings" after I click on the domain under Website Configuration.
    Screen shot attached below.



    Settings > Website Groups > Websites" data-title="ABP > Settings > Website Groups > Websites" width="200" data-modalsrc="https://s3.amazonaws.com/higherlogicdownload/IMPERVA/UploadedImages/NKZFg9CdTbimMM8Kspws_temp.png" data-imgbase="https://s3.amazonaws.com/higherlogicdownload/IMPERVA/UploadedImages/NKZFg9CdTbimMM8Kspws_temp.png" data-imgthumbnail="https://s3.amazonaws.com/higherlogicdownload/IMPERVA/UploadedImages/NKZFg9CdTbimMM8Kspws_temp-T.jpg" data-imgmedium="https://s3.amazonaws.com/higherlogicdownload/IMPERVA/UploadedImages/NKZFg9CdTbimMM8Kspws_temp-M.jpg" data-imglarge="https://s3.amazonaws.com/higherlogicdownload/IMPERVA/UploadedImages/NKZFg9CdTbimMM8Kspws_temp-L.jpg">


    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------



  • 6.  RE: ABP and F5 Connector

    Posted 09-15-2020 13:25

    Hi Derrick,

    I am in a time crunch and appreciate it if you can review my last post and help me enable Captcha for the site.

    I am also available all day today (PST timezone) if you would like to have a quick call and review my settings on Zoom.

    Thank you,



    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------



  • 7.  RE: ABP and F5 Connector

    Posted 09-15-2020 13:32
    Hey Muhammad,

    In your screenshot, where you clipped the domain name to censor it under Website configuration, click on the domain and it will bring up another window that has the advanced settings in it.

    Let me know how that goes.

    ------------------------------
    Derrick Lowder
    Tech Lead - ABP Operations
    Imperva, Inc
    ------------------------------



  • 8.  RE: ABP and F5 Connector

    Posted 09-15-2020 13:53
    Found it and enable "Geetest".

    Now I see a "captcha verification" button on "Pardon our Interruption" page.

    After clicking and verifying the Captcha ... it says verification succeeded but not redirecting back to "Origin" site ... Screen shot attached



    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------



  • 9.  RE: ABP and F5 Connector

    Posted 09-15-2020 14:13
    Same issue with reCaptcha v2 ... not redirecting back to "Origin" site.



    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------



  • 10.  RE: ABP and F5 Connector

    Posted 09-15-2020 16:16
    Are there any log entries in /var/log/ltm when you solve the captcha?

    If you inspect the page and go to the network tab, what response code do you see for the POST to the challenge path after the captcha is cleared?

    ------------------------------
    Derrick Lowder
    Tech Lead - ABP Operations
    Imperva, Inc
    ------------------------------



  • 11.  RE: ABP and F5 Connector

    Posted 09-15-2020 17:29
    There aren't any errors or log entries in /var/log/ltm ...

    I am seeing the challenge path it is referring to
    https://my.domainname.com/my-challenge-path

    Which is not valid for my application. I am verifying my application and will get back to you.

    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------



  • 12.  RE: ABP and F5 Connector

    Posted 09-16-2020 18:42
    hi Derrick,

    I couldn't able to figure it out at my end why the browser is not redirecting back to the Origin server after clearing the captcha.

    We heavily rely on Distil Networks and its sunsetting deadline fast approaching, can you please spare 30 mins for a Zoom call and review configuration as well as troubleshoot with me. I will make myself available whenever you are.

    I want to resolve this issue before the end of this week.

    Please let me know.

    Thank you,


    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------



  • 13.  RE: ABP and F5 Connector

    Posted 09-17-2020 14:34
    @Muhammad Shoaib, one of our support engineers just reached out. It should be in your inbox. Good luck, and feel free to post here what the resolution was.  ​

    ------------------------------
    Christopher Detzel
    Community Manager
    Imperva
    ------------------------------



  • 14.  RE: ABP and F5 Connector

    Posted 09-17-2020 16:06

    Thank you @Christopher Detzel for your response.

    Distil Support did reach out to me but I was already on the zoom call with Derrick. He helped me to resolve the Captcha issue.

    I have to add the following code our application iRules on BigIP

    if {$uri contains "my-challenge-path"} { return }


    Basically, telling the application iRule that don't execute if URI contains the "challenge path"



    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------



  • 15.  RE: ABP and F5 Connector

    Posted 09-17-2020 17:40
    F5 connector seems to be working on my test site and planning to release it for the live site soon ... 

    Next I need to find a way to configure it for all the sites. Let's say I have 500+ individual sites but all of them are the subdomains. 

    e.g. 

    client1.example.com
    client2.example.com
    client3.example.com
    ..
    ..
    client500.example.com

    Is there a way to configure standard policy, let say, on the root domain "example.com" so that it will protect all 500 sites.

    where as if client200 wants to have custom policy e.g. block specific User-Agent, or block certain IPs etc. then new policy will be created. The new policy will take all the standard settings from standard policy but override specifics.


    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------



  • 16.  RE: ABP and F5 Connector

    Posted 09-17-2020 18:17
    After looking into it. I was informed that the apiKeyId and analysisHost address are set by account, so those aren't a problem, however, the tokenEncryptionKey is set by website group. The good news is you can open a support ticket and have the tokenEncryptionKeys manually updated so that they are the same. This way you can create as many website groups as you want and the connector config settings will be the same.

    There is a feature request in to allow end users to update these keys so that a support ticket isn't needed to sync them. I don't have an ETA on that at the moment.

    Let us know if you have any other issues.

    ------------------------------
    Derrick Lowder
    Tech Lead - ABP Operations
    Imperva, Inc
    ------------------------------



  • 17.  RE: ABP and F5 Connector

    Posted 09-22-2020 19:29
    Derrick,

    I have only one domain in testing .. I have started configuring the production unit so that I have multiple sites and then I can test the scenario you have mentioned above.

    While setting up the production, BigIP Unit TMM ( Traffic Manager ) crashed. I opened a ticket with F5 and the confirmed a bug in the version.

    They recommended to upgrade it to 13.1.3.4.

    I will post here after the upgrade.

    Thank you


    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------



  • 18.  RE: ABP and F5 Connector

    Posted 09-22-2020 21:29
    Muhammad,

    Thanks for opening the ticket with F5 and confirming that bug. Did they happen to link you to a bug report on F5's dev central or any other publicly available documentation for the bug?

    I'll make sure to get our documentation updated with that version information.

    Thanks for following up and keep us updated with your progress.


    ------------------------------
    Derrick Lowder
    Tech Lead - ABP Operations
    Imperva, Inc
    ------------------------------



  • 19.  RE: ABP and F5 Connector

    Posted 09-23-2020 01:20
    Here is the bug link :

    Bug ID 696908: Updating iRule causes TMM to crash


    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------



  • 20.  RE: ABP and F5 Connector

    Posted 09-30-2020 14:58
    Production BigIPs are upgraded to 13.1.3.4.

    One client site is live on it and seems to be stable.

    ------------------------------
    Muhammad Shoaib
    Sr. Manager NRE
    Paciolan Inc.
    Irvine CA
    ------------------------------