The conventional pre-history of Japan involves horseback riding soldiers called the Yayoi entering Japan around the area where it comes closest to Korea around sometime in the middle of the first millenium before the current era, where the indigeneous people, who had lived there for 30,000 years, called the Jomon, a fishing oriented pottery making people who were linguistic and ethnic close relatives of the modern Ainu minority of Northern Japan were assimilated into the Yayoi superstrate in a process of ethnogenesis that gave rise to the modern Japanese people.
Until around 1000 CE, the northern Honshū island was inhabited by the Emishi to whom we can be quite definitive in attaching an Ainu ethnicity and language affiliation, and Ainu related people also inhabited the region known as Ezo, consisting of the island of Hokkaidō; and were formerly spoken in southern and central Sakhalin, and the Kuril Islands. So, for the first 1300 years, the Yayoi were pretty much confined to Southern Japan.
The Yayoi may have been some manner of Korean people, although which of the ancient Korean kingdoms they (and the modern Korean language) have the strongest affinity to is disputed.
As I noted in a wikipedia article on the origin of the Japanese language that is linked above: "Current estimates are that "wago" (i.e. words attributable to the original Yaoyi language) make up 33.8% of the Japanese lexicon, that "knago" (i.e. words with roots borrowed from Chinese since the 5th century CE) make up 49.1% of Japanese words (and in addition, the Chinese ideograms used in the Japanese written language), that foreign words called gairaigo make up 8.8% of Japanese words, and that 8.3% of Japanese words are konshugo that draw upon multiple languages. This account attributes only a small number of words in modern Japanese to Ainu roots." Almost all of the various characters used to write down Japanese are also borrowed from Chinese.
I'd instinctively favor an account that suggests that the Ainu linguistic contribution is undercounted, but because Ainu is a living language that is well attested, as is Chinese, and given that Japanese and Chinese have been a literary language for most of the time period in question, I suspect that an identification of wago words with the original Yayoi language is probably pretty accurate.
There are strongly suggestive indications that the Yayoi language was Altaic, a language family that also has as core members, the Turkic languages, the Mongolian languages, and the Tungusic languages of Manchuria. The geographic scope of the proposed Altaic language family members is considerable as this map from Wikipedia indicates:
But, historically, this spread was quite recent.
The Turkic languages reached Central Asia, Europe and Turkey within the last seventeen hundred years or so. Their ethnogenesis is associated with the arrival of the Bronze Age and horses in the Northeastern Asian region sometime in the vincinity of 1500 BCE to 700 BCE.
The traditional date for the Yayoi arrival in Japan is 2700 years ago, and most evidence to date suggest an actual arrival a few hundred years later. This would be consistent with the Yayoi as a cultural offshoot of the proto-Turkic peoples.
While the empire of Ghengis Khan was the largest on land known to man, in the late middle ages, until a eight or nine hundred years ago, it was confined to Mongolia, Northern China and perhaps Eastern Siberia where it is found today (the only linguistic legacy of its expansion is the Oirat language spoken by peoples on the Caspian plains of Russia. The Mongolians first entire recorded history in the 4th century CE in Manchuria and Mongolia as a nomadic people called the Khitan people.
The Tungusic languages have remains at all times largely confined to Manchuria.
Japanese and Korean are clearly not Tibet-Burmese languages, despite their heavy borrowing from Chinese, and the linguistic case of the an Altaic affiliation for Japanese and Korean (presumably via its Yayoi roots) is not trivial. The Altaic languages, Old Japanese and Korean all have a linguistic feature called vowel harmony that is rarely, if ever, found in other languages. Similarities have also been found in verb conjugation classes, and there has been some success in matching tonal aspects of Japanese to consonants found in other Altaic languages. Vovin has suggested that as many as 15 to 23 core Old Japanese words have Altaic proto-language counterparts, far more than chance would suggest, and almost all of the Altaic-Japanese correspondences are wago words. The Altaic languages, Japanese, and Korean are also notable for a lack of noun classes and lack of grammatic gender.
So far, so good. There is a problem, however. While Japanese population genetics do parse rather neatly into East Asian and Jomon components, the East Asian genetic component looks far more like the Chinese than it does like the Altaic populations of Northeastern Eurasia.
It isn't so hard to imagine how this could happen. In the same vein, Turkic peoples of Northeast Eurasia don't have a very strong genetic link to the Turkish language speaking people of Anatolia. One imagines that an ethnically Chinese population in and around Korea may have been a subject people of a Altaic language speaking elite leading to a language shift in the population (at least among the elites that would go on to conquer Japan as the Yayoi), with little Altaic genetic admiture from the ruling elite to the soldiers who went on to conquer Japan.
This pushes the genetic forebears of the Yayoi to the Southwest, and the linguistic and cultural forebears of the Yayoi to the North, probably North of Korea entirely.
Given that Japanese appears to have more lexical similarity to Proto-Turkic than to other Altaic language family languages, that Proto-Turkic was the first of the Alataic languages to experience an expansion out of Asia, and that Japanese lexical links to Altaic languages appear to be mediated through Proto-Turkic, one imagines the linguistic forebears of the Japanese to be Proto-Turkic peoples of Northeast Eurasia.
This would have been some time after the 4th millenium before the current era, when the horse appears to have been domesticated (probably reaching the Turks as a cultural transmission from Indo-European language speakers such as the Tocharians or their immediately predecessors in Central Asia), but no later than the 3rd millenium before the current era when the Yayoi arrive in Japan, which is strongly consistent with Proto-Turkic ethnogenesis.
The Proto-Turkic expansion would presumably start a little later than the Indo-European expansions, given the direction of horse domestication transmission and Bronze Age technology transmission across the Russian Steppe, and Finno-Urgic a.k.a. Uralic languages of that region appear to be older still than Indo-European languages. In addition, some of the region, prior to the expansions of Indo-European languages and Altaic languages with the horse would have spoken Paleo-Siberian languages such as Yenesian which has linguistic links to the Na-Dene languages of North America.