To catch any monster (except Humanoids, Elites and Bosses, these can't be tamed) you'll need to have in your bag and quick/action slot bar some traps. These vary in their capture chance and some are specially made to capture sea pets, air pets, etc.

Each time you use a trap, be it a success or not, 1 trap will be consumed.

The capture chances rise as the monster HP drops.

If you capture the pet, you have now in your bag a Soul Orb Itemicon 97.

To get your pet you must have it Appraised. You'll need a Pet Appraisal book, which can be bought at the pet merchant. Then just double left click the Soul ball and you have your pet ready at the Pet menu. When you appraise a pet its talent level varies. It means that you cannot have a pet that is a supreme star 2 100%. their ranks are Garbage, Clumsy, Regular, Smart, Genius Star 1, Genius Star 2, Genius Star 3, Supreme Star 1, Supreme Star 2, Supreme Star 3 and Ultimate. You can have a small chance to capture a supreme star 1,2,3 but for Ultimate pet is nearly impossible.

Itemicon 141 Primary Trap Traps that have a lower chance to capture monsters.
Itemicon 142 Medium Trap Traps that have a medium chance to capture monsters.
Itemicon 143 Master Trap
Itemicon 142 Animal Trap Traps that are specially designed to capture animal-kin monsters.
Itemicon 142 Net Trap Traps that are specially designed to capture aquatic monsters.
Itemicon 85 Pet Appraisal Book Used to identify a Soul orb. Consumed upon use.

Note : 1 Soul Orb occupies 1 bag slot, 10 Appraisal Books (same type) occupy 1 bag slot, 20 trap (same type) occupy 1 bag slot.