Jasper is right, it messes up balance and whole match if afk player has to be speced after the start.
There is an easy solution to this, someone already mentioned it: Automatically set Spectate mode after ~ 3 minutes in lobby, so everyone must renew their Ready (Play) status. This is the only way to keep players really ready.
To avoid unwanted starting just in the second when you were speced, implement some kind of warning or countdown.