RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemm( handle)` with GPU only

We Are Going To Discuss About RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemm( handle)` with GPU only. So lets Start this Python Article.

RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemm( handle)` with GPU only

  1. How to solve RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemm( handle)` with GPU only

    Please know that, it can also be caused if you have a mismatch between the dimension of your input tensor and the dimensions of your nn.Linear module. (ex. input.shape = (a, b) and nn.Linear(c, c, bias=False) with c not matching).

  2. RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemm( handle)` with GPU only

    Please know that, it can also be caused if you have a mismatch between the dimension of your input tensor and the dimensions of your nn.Linear module. (ex. input.shape = (a, b) and nn.Linear(c, c, bias=False) with c not matching).

Solution 1

Please know that, it can also be caused if you have a mismatch between the dimension of your input tensor and the dimensions of your nn.Linear module. (ex. input.shape = (a, b) and nn.Linear(c, c, bias=False) with c not matching).

Original Author Loich Of This Content

Solution 2

With searched with the partial keywords, I finally got the similar situation.
Because of the stability, I used the CUDA 10.2 version. The reference asked to upgrade CUDA toolkit to higher – 11.2 in my case – and problem solved!
I’ve deal with other training processes but this one only caused error. As the CUDA error occurred with various reasons, changes the version could be counted for solution.

Original Author Loich Of This Content

Solution 3

Rightly said by Loich, and I think shape mismatch is a prime reason why this error is thrown.

I too got this error while training a image recognition model, where the shapes of – output of final Conv2d and input of first Linear layers was not same.

If none of that works, then the best thing to do is to run a smaller version of the process on CPU and recreate the error. When running it on CPU instead of CUDA, you will get a more useful traceback that can solve your error.

One remedy explained in this answer (quoted above) is, with disabled gpu try to recreate similar situation by executing the code (without changing any line) on cpu, it should give better and understandable error.

P.S.: Although, the original question states that their code is executing fine on cpu, I’ve posted this answer for someone with similar error and not as a result of Cuda version mismatch.

Original Author theProcrastinator Of This Content

Conclusion

So This is all About This Tutorial. Hope This Tutorial Helped You. Thank You.

Also Read,

ittutorial team

I am an Information Technology Engineer. I have Completed my MCA And I have 4 Year Plus Experience, I am a web developer with knowledge of multiple back-end platforms Like PHP, Node.js, Python and frontend JavaScript frameworks Like Angular, React, and Vue.

Leave a Comment