My native language is also not English. By undergrad, I struggled with spoken English. However, in workplaces, it became inevitable to do without good English so these are the things I did. Watch good quality English language news channels. Watch Good movies. Most importantly talk to people. Tell your friends or family that you want to improve your language and then talk to them in this language. that will improve your spoken as well as give you confidence.