Sex deprived because Non-ADHD spouse can't trust me

My husband was diagnosed with diabetes about four or five years ago. I don't know if this is the reason for low sex drive or not, but when I try to talk to him about it he blames it on not wanting to have sex with me because not interested in me anymore. Due to my forgetfulness of items that i have forgotten to do, packed in his lunch, errands I didn't run, didn't pick up on knowing what types of things, food or whatever he would like for me to buy for him etc. We have been together for twenty years and there are times that I didn't think we would make it. I miss intimacy. He tells me he loves me everyday, but I cannot get him interested in sex. He buys me "toys" and tells me that it is my problem.