I felt
exactly the same when I first thought about learning to drive
I was terrified of other people on the road and how I would ever start the car up a hill!! lol
It took me 6 months to get the courage to take my written test, which like PinkPiglet I ended up passing first time!! lol
In my first lesson I was shaking I was so scared but my instructor was really really great, he was so patient and talked me through everything and now I've had about 30 lessons and I'm taking my test on the 4th Febuary

All I can really say is yeh it is scary but the scaryness DOES go away when you get into it and the most important thing for me was finding the right instructor, don't be afraid to try a few out until you find the right one.
Also, as someone else said I would say don't go with AA or RAC because I've heard they go really slow so you'll have more lessons.
You could always have a few lessons and then if you're still not feeling any better then just leave it?
Although honestly, if I can do it then you can hun!!
Good luck

xx