This will be a simplistic answer since I don't work in the phone industry but this is what I know:
First, we must assume that we are leaving the newer kinds of phone service out of the equation - I might touch on them later. Anyway, yes, in the traditional phone service setup, all phones are connected by lines & different areas are connected by cables that hold what we could term as super lines...basically a larger collection of lines to allow for more data flow back & forth. The phone service reaches your house through a line actually connected to your house above or below ground. In order to get there, it winds through your neighborhood & connects to a type of juncture box which houses all of the equipment for the area (these are normally short-ish green boxes on the side of the road or look like transformers at the top of telephone poles).
As for connecting countries themselves, there is/was (not sure if it is still in use but I think it is) something called the TransAtlantic Cable which originally carried the ability for telephone service overseas. Of course, when you make the jump to another continent, you then continue with the lines & cables either above or below ground.
Back in the day, "Ma Bell" (nickname, can't think of the real name right now) & AT&T owned all the lines. At some point, this company was split up into what became known as the "baby Bells" which included, Southern Bell, Pacific Bell, etc. The other early player in the game & in our ability to communicate across the country & overseas, AT&T, is a company you are likely still familiar with today.
I would suggest researching both "AT&T", "Ma Bell" & something like "history of telephone communications" online to get more information as the history of the worldwide telephone service is quite interesting. You'll be amazed at what folks accomplished when all of this was first set up; I certainly am when I look at where we were even when I was a child as opposed to where we are today.
As for my comment on the newer technologies, those would include broadband (or internet) telephone service - which carries the information over either the DSL or cable connected to your computer - & satellite telephone service. I can't say much to satellite as I've not researched it...