Upgrades and items have two main ways of enhancing your damage: base damage and power scaling, which is either attack or skill power.
Calculating damage dealt (enemy resistance not considered) = (Base Damage + attack add) * attack power multiplier * general damage multiplier
5 base damage primary, +3 damage from spiked flail, with 20 attack power = (5+3)*(1+20*0.2)= 8*1.4= 13.2 damage dealt
30 base damage spell with 40 skill power = 30*(1+40*.02) = 30*1.8 = 54 damage dealt
Items that increase base damage (physical or magical), attack power, or mentions it is primary exclusive only increase the strength of your primary attack.
Items that increase skill power, skill crit chance, skill crit multiplier increase the damage of every damage source that is NOT a primary attack, it will even enhance the base damage of passives such as Paladin Juggernaut and Priest Celestial Orbs.
Armor penetration and magic penetration is specific to the damage type of your skill or primary attack. Note that hybrid damage does half physical and half magical damage so primaries and skills will only receive 50% effective benefit out of penetration items/buffs.
Critical chance is calculated a bit strangely in this game. Each source of crit chance is independently rolled for either a positive or a negative outcome to determine if it crits or not. If you have two sources of crit and you roll both crits for a positive outcome you can get 4x damage.
For example, if you have two items and one gives 20% crit chance and another gives 10% crit chance, 20% of the time you do 2x damage, another 10% of the time you do 2x damage, (20%*10%)=2% of the time you might do 4x damage!
Damage output = (Base Damage)(1 + Critical damage mod) * N, N=number of positive rolls from different sources of crit
e.g. 50 base damage, 4 sources of crit but only 3 rolled a positive event, 50% bonus crit damage
Damage output = (50)(1+1.5)*3 = 375 damage