Well, you need enough memory to have 2 or 3 turns worth of spells available since you won't be using autoattacks after the start. Add a few buff and mobility spells and you'll want to have at least 20 memory slots. At level 20, you'll have 13 base slots so getting to 20 memory will give you 23 slots total. That should be enough.

Intelligence should be maxed ASAP for pure damage. So the progression is to get only enough memory to get the spells you must have and the rest into INT. After that add wits for initiative and crit chance using the savage sortiledge talent.