Assistance Requested: Integrating QAT with StrongSwan on Linux Kernel - Counters Not Incrementing

[lucasvp]

Hello, QAT Community,

I am currently working on integrating QAT (Quality and Acceleration Technology) with StrongSwan on the Linux kernel and have encountered a challenge that I hope you can help me overcome. We have successfully loaded the necessary modules for QAT operation, but we are facing an issue where, despite what seems to be a correct configuration, QAT is not being utilized when generating traffic through StrongSwan.

Here are the details of our current setup:

Loaded QAT Modules:

qat_api 565248 0
qat_c3xxx 20480 0
intel_qat 270336 3 qat_api,usdm_drv,qat_c3xxx
We confirmed that the modules are correctly loaded by using the lsmod | grep qat command, which returns the list of modules above.

Issue:
Despite the modules being loaded, when we generate traffic expecting QAT to be utilized, we observe that the counters located in /sys/kernel/debug/qat_c3xxx_0000:01:00.0/fw_counters do not increment, indicating that QAT is not effectively being used.

Steps Already Taken:

Verification of the correct loading of QAT modules.
Review of StrongSwan configuration to ensure it is set up to use hardware acceleration.
Checking system and QAT logs for potential errors, to no avail.
I am seeking guidance, insights, or shared experiences that might help me identify and solve this issue. Any advice on how to ensure that QAT is correctly utilized with StrongSwan would be immensely appreciated.

Thank you in advance for any support you can offer!

Sincerely,
Lucas Pereira

[lucasvp]

Dear Tobias,

We made the QAT works here, with the Intel's support.

---

Hi Joel,

I hope this message finds you well. Thank so much for you help!!

Indeed, I had not applied any configure flags during the build process, which, as you correctly anticipated, resulted in the command cat /proc/crypto | grep qat yielding no output. Following your advice, I rebuilt the driver with the configure flag --enable-qat-lkcf and it worked!!

The system is now operating as intended. Data transfer through the tunnel is functioning smoothly, with the fw_counters showing incremental activity!

I cannot thank you enough for your invaluable assistance and expert guidance throughout this process. Your support was crucial in navigating these technical challenges.

Best regards,

Igor de Souza Santos
Líder de Desenvolvimento Junior 2

+55 11 2165-8888 | R:8861
www.blockbit.com

Esta mensagem pode conter informação confidencial ou privilegiada, sendo seu sigilo protegido por lei. Se você não for o destinatário ou a pessoa autorizada a receber esta mensagem, não pode usar, copiar ou divulgar as informações nela contidas ou tomar qualquer ação baseada nessas informações. Se você recebeu esta mensagem por engano, por favor, avise imediatamente ao remetente, respondendo o e-mail e em seguida apague-a.Agradecemos sua cooperação.

This message may contain confidential or privileged information and its confidentiality is protected by law. If you are not the addressed or authorized person to receive this message, you must not use, copy, disclose or take any action based on it or any information herein. If you have received this message by mistake, please advise the sender immediately by replying the e-mail and then deleting it. Thank you for your cooperation.

De: Schuetze, Joel D joel.d.schuetze@intel.com
Enviado: sexta-feira, 5 de abril de 2024 10:01
Para: Lucas Vicente Pereira lpereira@blockbit.com
Cc: Igor de Souza Santos issantos@blockbit.com
Assunto: RE: Assistance Requested: Integrating QAT with StrongSwan on Linux Kernel

Você não costuma receber emails de joel.d.schuetze@intel.com. Saiba por que isso é importante
Hi Lucas,

Thank-you for providing these details. Much appreciated!

From these steps, it looks like we may be missing important configure flag for the QAT driver:

Here is excerpt from the Getting Started Guide where configuration flags are presented.

Without this flag, we would not be registering crypto algorithms with LKCF.

We can verify this with this command:

cat /proc/crypto | grep qat

I suspect if we ran this command prior to reinstalling QAT driver with this flag set we would not see output from the above command.

Please let me know if this addresses the issue observed.

Cheers,

Joel

From: Lucas Vicente Pereira lpereira@blockbit.com
Sent: Thursday, April 4, 2024 5:26 AM
To: Schuetze, Joel D joel.d.schuetze@intel.com
Cc: Igor de Souza Santos issantos@blockbit.com
Subject: RE: Assistance Requested: Integrating QAT with StrongSwan on Linux Kernel

Dear Joel, good morning

Thank you for your response.

I am trying to offer all informations:

---

cat versionfile
PACKAGE_TYPE=QAT

PACKAGE_OS=L

PACKAGE_VERSION_MAJOR_NUMBER=4

PACKAGE_VERSION_MINOR_NUMBER=24

PACKAGE_VERSION_PATCH_NUMBER=0

PACKAGE_VERSION_BUILD_NUMBER=00005

---

We used this package: https://downloadmirror.intel.com/795697/QAT.L.4.24.0-00005.tar.gz

We compiled from:

driver_install.sh from repo https://github.com/intel/QAT_Engine.git, branch master.

--

Conf is the /etc/c3xxx_dev0.conf:

ServicesProfile = DEFAULT

 

ConfigVersion = 2

 

#Default values for number of concurrent requests*/
CyNumConcurrentSymRequests = 512
CyNumConcurrentAsymRequests = 64

 

#Statistics, valid values: 1,0
statsGeneral = 1
statsDh = 1
statsDrbg = 1
statsDsa = 1
statsEcc = 1
statsKeyGen = 1
statsDc = 1
statsLn = 1
statsPrime = 1
statsRsa = 1
statsSym = 1

 

# This flag is to enable device auto reset on heartbeat error
AutoResetOnError = 0

 

##############################################
# Kernel Instances Section
##############################################
[KERNEL]
NumberCyInstances = 1
NumberDcInstances = 0

 

# Crypto - Kernel instance #0
Cy0Name = "IPSec0"
Cy0IsPolled = 0
Cy0CoreAffinity = 0

 

##############################################
# User Process Instance Section
##############################################
[SHIM]
NumberCyInstances = 6
NumberDcInstances = 2
NumProcesses = 1
LimitDevAccess = 0

 

# Crypto - User instance #0
Cy0Name = "IPSEC0"
Cy0IsPolled = 1
# List of core affinities
Cy0CoreAffinity = 0

 

# Crypto - User instance #1
Cy1Name = "IPSEC1"
Cy1IsPolled = 1
# List of core affinities
Cy1CoreAffinity = 1

 

# Crypto - User instance #2
Cy2Name = "IPSEC2"
Cy2IsPolled = 1
# List of core affinities
Cy2CoreAffinity = 2

 

# Crypto - User instance #3
Cy3Name = "IPSEC3"
Cy3IsPolled = 1
# List of core affinities
Cy3CoreAffinity = 3

 

# Crypto - User instance #4
Cy4Name = "IPSEC4"
Cy4IsPolled = 1
# List of core affinities
Cy4CoreAffinity = 4

 

# Crypto - User instance #5
Cy5Name = "IPSEC5"
Cy5IsPolled = 1
# List of core affinities
Cy5CoreAffinity = 5

 

# Data Compression - User instance #0
Dc0Name = "Dc0"
Dc0IsPolled = 1
# List of core affinities
Dc0CoreAffinity = 0

 

# Data Compression - User instance #1
Dc1Name = "Dc1"
Dc1IsPolled = 1
# List of core affinities
Dc1CoreAffinity = 1

 

##############################################
# User Process Instance Section
##############################################
[SSL]
NumberCyInstances = 6
NumberDcInstances = 2
NumProcesses = 1
LimitDevAccess = 0

 

# Crypto - User instance #0
Cy0Name = "SSL0"
Cy0IsPolled = 1
# List of core affinities
Cy0CoreAffinity = 0

 

# Crypto - User instance #1
Cy1Name = "SSL1"
Cy1IsPolled = 1
# List of core affinities
Cy1CoreAffinity = 1

 

# Crypto - User instance #2
Cy2Name = "SSL2"
Cy2IsPolled = 1
# List of core affinities
Cy2CoreAffinity = 2

 

# Crypto - User instance #3
Cy3Name = "SSL3"
Cy3IsPolled = 1
# List of core affinities
Cy3CoreAffinity = 3

 

# Crypto - User instance #4
Cy4Name = "SSL4"
Cy4IsPolled = 1
# List of core affinities
Cy4CoreAffinity = 4

 

# Crypto - User instance #5
Cy5Name = "SSL5"
Cy5IsPolled = 1
# List of core affinities
Cy5CoreAffinity = 5

 

# Data Compression - User instance #0
Dc0Name = "Dc0"
Dc0IsPolled = 1
# List of core affinities
Dc0CoreAffinity = 0

 

# Data Compression - User instance #1
Dc1Name = "Dc1"
Dc1IsPolled = 1
# List of core affinities
Dc1CoreAffinity = 1

 

---

[root@lucas-node2 QAT_Engine]# adf_ctl status
Checking status of all devices.
There is 1 QAT acceleration device(s) in the system:
qat_dev0 - type: c3xxx, inst_id: 0, node_id: 0, bsf: 0000:01:00.0, #accel: 3 #engines: 6 state: up

Show status from cli "adf_ctl"

also, test cli with cpa_sample_code, when I use this command the "fw_counters" increments correctly...

---

Thank you for your help.

Sincerely,

Lucas Pereira

Chief Technology Officer
+55 11 2165 8888 | R:9999
+55 12 9 8232 9068
www.blockbit.com
Forma

Descrição gerada automaticamente com confiança baixa Forma

Descrição gerada automaticamente com confiança baixa

Esta mensagem pode conter informação confidencial ou privilegiada, sendo seu sigilo protegido por lei. Se você não for o destinatário ou a pessoa autorizada a receber esta mensagem, não pode usar, copiar ou divulgar as informações nela contidas ou tomar qualquer ação baseada nessas informações. Se você recebeu esta mensagem por engano, por favor, avise imediatamente ao remetente, respondendo o e-mail e em seguida apague-a.Agradecemos sua cooperação.

This message may contain confidential or privileged information and its confidentiality is protected by law. If you are not the addressed or authorized person to receive this message, you must not use, copy, disclose or take any action based on it or any information herein. If you have received this message by mistake, please advise the sender immediately by replying the e-mail and then deleting it. Thank you for your cooperation.

De: Schuetze, Joel D joel.d.schuetze@intel.com
Enviado: quarta-feira, 3 de abril de 2024 17:13
Para: Lucas Vicente Pereira lpereira@blockbit.com
Cc: Igor de Souza Santos issantos@blockbit.com
Assunto: RE: Assistance Requested: Integrating QAT with StrongSwan on Linux Kernel

Some people who received this message don't often get email from joel.d.schuetze@intel.com. Learn why this is important

Hi Lucas,

Thanks for reaching out! Can you confirm the QAT driver that is being used?

Are we using in-tree QAT kernel modules? Or were they installed from QAT package?

Thanks,

Joel

Atenciosamente,