It's easy to forget that jazz wasn't born in a university. Improvisation is just that. It's you telling someone else something via sounds and albeit emotion. It's a language. You can only convey a message as clearly as you have a grasp on the medium it's being spoken.
Improv is the best form for expressing emotion in real time. Indian Classical is 90% improv, but conveys deep deep emotion through spontaneous music. At a level almost impossible with equal temperament. But it requires things not taught in western theory. It requires a respect and kind of spiritual attachment to the notes, unlike western music that focuses on cold mathematical structure. It requires training. Not practice, but training. Practice is a sort of mindless drilling, while training is in depth playing and improvising and exploring. The difference between a scale/mode and raga is vast, even though they may have the same pitches. Find out why.
What made Miles so great (as well as jaco and herbie hancock and others) was that he was able to give meaning to the notes, rather than just trying to reach a harmonic ideal like for example coltrane. Coltrane was a fabulous technical player, but so technical with so many flurries of notes that the notes become meaningless, like he's just shredding without feeling any of the notes, and trying to create complex harmonies with a monophonic instrument. With miles, every note has depth and feeling behind it, like in Indian music. Indian music ties justly intonated intervals to the mythology of chakras, which gives a certain importance or sanctity to each note, and play in only a single key, without any functional harmony. Rhythm, melody, and inflection are what counts, even in western music. The voice is still the most popular instrument all over the world, and it's monophonic.
Improv should be about emotional and spiritual communication, rather than just a mathematical puzzle in real time. That's why I like to focus on a few notes, and learn how to make them work. I can play just 2 notes and make it interesting with rhythm and inflection. A lot of bass and guitar teachers talk about "mastering the fretboard" as if that's the way to make good music. I'd say it's much more important to "master" making a few notes sound interesting, and giving depth to simplicity. If you can do that, everything else is easy. But it's not easy to do. The mind always wanders, and the fingers always want to move, and you always want to hear yourself. So you have to:
1. constantly be in the moment. Do not think about the past or future while improvising. The mind always has to be focused on something, so when it starts to wander, learn to catch yourself, and if you can't think of anything else, think about the instrument itself, how it feels in your hands, or the relationship between the strings and pickups, or anything related to the instrument, rather than thinking about picking up your kid from school the next day or what you are cooking for dinner that night.
2. Don't let yourself space out. Train yourself to always be aware, and train people you are playing with to give each other cues when anyone gets the dreaded blank stare. Cues such as like eye movements, hand gestures, loud accents, etc. Everyone spaces out, but the really great musicians know how to control it, only letting themselves space when it's beneficial, and able to snap out of it at will
3. Always be aware of rhythmic space. Don't always play constantly all the time, note after note. The more rests and space you have, the more meaning the notes have
4. Change up note durations. Practice playing the same lines staccato and legato and mix up note lengths, even with the same lines, this makes it more interesting
5. Play in front of a mirror. Watch yourself, your movements, the way you look playing, your facial expressions. Facial expressions are an important part of being a musician. You have to look like you know what you are doing, having fun, and whatever other feelings you want to project to the audience and musicians. The mindless "cheese" face that comes from not being aware of your demeanor makes you look like a buffonish porn star, and it happens to everyone. But maybe that's what you are going for, who knows.
6. Mixing. I only really started to learn to dynamically play bass and music in general after I began engineering, specifically mixing. So many musicians including myself in the past simply don't listen. They are just listening to themselves with the rest of the band in the background. Simple balance between instruments is a lot harder than most people think, especially in real time. But you have to mix constantly. Every musician has to mix themselves with the rest of the band, which is extremely difficult to do while playing and listening, yet supremely important. Having a drummer that controls their dynamics is key. This is why conductors are so important to orchestras. You have to have dynamic control, and it's better to have someone, preferably a disciplined drummer, to lead the dynamics of the music. Otherwise it's just a competition for loudness, and because drummers are the loudest and have impaired hearing anyway, they will always win. Hence the endless smashing of music and the loudness wars
7. Quiet playing/passages often garner more attention than loud ones, because people have to kind of "lean in", as it were, to hear it, focusing their attention. Good dynamic playing makes use of extremely loud and extremely quiet passages (and everything in between) tastefully juxtaposed
8. Don't be afraid to speed up and slow down, as long as everyone is phase locked in rhythm, especially on the downbeat. In fact, you can do anything rhytmically, polyrhytmically, etc, and as long as you come back on the 1, it's all good. OCD has made people think that perfect time is the ideal, when rushing and lagging (at least slightly, not necessarily enough to notice) makes music more interesting. "Grid" thinking/playing causes robotic lifeless grid music.
9. Embrace mistakes. Mistakes are a huge part of improv. If you are not making mistakes, then you aren't brave enough to take chances. But learn to not let on that you made a mistake. Usually when i see a musician make a mistake, they usually have an awkward smile, and you know they messed up, when most of the time you wouldn't know. Don't do that. Try to make the mistake into something musical, even repeating the mistake to make it look like you meant to do it. Poker face.
10. I approach the fretboard sometimes like a chess board. Think about new ways to move around, rather than getting stuck in the same patterns. This is different from mastering the fretboard. The point is to not master it, to never think you know it. There's an infinity of possibilities, it's up to you to unlock new paths. Think about moving sideways and diagonal melodically rather than just up and down, ascending and descending. Yes, that makes no sense, but do it anyway.