You may not always begin the prayer with the imām. There are various matters that you need to be aware of relating to lateness and making up parts of the prayer.
The most important thing in joining the congregational prayer late is that you should perform the takbeerat-ul-iḥrām (the opening takbeer). Without this, the prayer is not valid. So, before joining the action which the imām is doing, you should stand in the appropriate place, raise your hands to shoulder or ear level, and say: 'Allaahu Akbar'. Then, you should immediately join the imām in the action that he is doing. So, if he is prostrating, you should immediately prostrate, without doing any of the actions that come before. You should not wait for the imām to move, or wait for him to stand, as some people do. This is because of the statement of the Prophet (may the peace and blessings of Allah be upon him):
"The imām has only been made to follow, so when he says Allaahu Akbar, then say Allaahu Akbar; and when he bows, then bow; and when he says, sami' allaahu li-man ḥamidah, then say, rabbanaa wa lakal-ḥamd; and when he prostrates, then prostrate; and if he prays sitting, then pray sitting altogether." [al-Bukhārī: 734]
If there is only one person praying, and you want to join the prayer, it is important that the person knows and accepts that they are going to be leading you in prayer. The simplest way to achieve this is to tap them gently on the shoulder, so that they are aware that they are going to be leading someone in prayer. However, if they show that they are aware of this, by making space for you, or something similar, then this is also acceptable.
It was narrated that Ibn 'Abbās (may Allah be pleased with him) said:
"I stayed overnight with my maternal aunt. The Prophet (blessings and peace of Allah be upon him) got up to pray at night and I got up to pray with him. I stood on his left and he took hold of my head and made me stand on his right." [Muslim: 763]
As mentioned in the arrangement of the rows, two people stand next to each other, and three people stand with the imām in front. If two people are praying and you wish to join them, simply tap the person who is following the imām on the shoulder, and they will move backwards in their prayer to form a row behind the imām.
It was narrated that Anas (may Allah be pleased with him) said:
"The Messenger of Allah (may the peace and blessings of Allah be upon him) was praying in Ramaḍān, and I came and stood beside him, and another man came and stood too, until there was a group of us. When the Prophet (blessings and peace of Allah be upon him) realized that I was behind him, he made his prayer brief." [Muslim: 1104]
When joining the imām late, it is important to count the number of rak'ah (units of prayer) that you pray with the imām, so that you know how many to make up. A rak'ah is counted as long as you catch the imām in rukoo' (the bowing position), or before. So, if you come into the masjid, and the imām is bowing, and you say the opening takbeer, and then join the imām in the bowing position, then this rak'ah is counted as a full unit of prayer, even though you may have missed a part of it. The evidence for this is the narration of Abū Bakrah (may Allah be pleased with him) that:
"He came to the Prophet (may the peace and blessings of Allaah be upon him) when he was bowing and he bowed with him, before he reached the row. He mentioned that to the Prophet (may the peace and blessings of Allah be upon him) and he said: 'May Allah make you more keen, and do not do it again.'" [al-Bukhārī: 783]
The words 'do not do it again' in this narration refer to him bowing before the row and shuffling forward, and not the fact that he caught the imām bowing. The evidence for this is that the Prophet (may the peace and blessings of Allah be upon him) didn't command him to repeat the prayer, or to make up an extra rak'ah, as well as the fact that he bowed with the imām in the first place, showing that he had an understanding that this would count as making up the entire unit of the prayer.
There is further evidence for this from the narration of Ibn Mas'ood (may Allaah be pleased with him) that he said:
"Whoever does not catch up with the imām when he is bowing has not caught up with that rak'ah." [al-Bayhaqi]
Therefore, if you catch the imām while he is bowing in the first rak'ah, there is nothing to make up from the prayer, and you finish the prayer with the imām. However, if you join the prayer later than that, you have to count how many rak'ah you perform with the imām, and whatever is missing at the end you have to make up.
You must follow the imām in everything that he does, up until the tasleem, when he says: 'Assalaamu 'alaykum waraḥmatullaah'. At this point, wait until he finishes saying it to both sides, and do not follow him in doing so; rather, you should stand up without saying the tasleem.
The key to making up the missing number of rak'ah is to concentrate on the number that you have prayed, and then to imagine that you are in that position in your regular prayer. For example, if you join the imām for Dhuhr and you only caught one rak'ah with the imām, stand up as though you are in your second rak'ah. That means that you are going to sit for the middle tashahhud in this rak'ah, then you are going to stand for your third and fourth rak'ah, just like you would in your normal prayer.
To give another example, if you join the imām for Maghrib prayer, and you join him in the second rak'ah, you would sit for the middle tashahhud immediately (because you follow the imām in everything that he does). You would then stand up for your second rak'ah and the imām's third, and would sit for the final tashahhud with the imām. You would then stand up for your third rak'ah, and would sit for the final tashahhud at the end of it (as is the norm in the third rak'ah of Maghrib).