You can dig straight down, but you should create a 2-block wide and 1-block long gap. You stand on 1 of the blocks and mine the other block. Then, you switch to the block that just got mined and mine the block you were standing on. Repeat.
At the same time, Minecraft is a sandbox game, meaning you can do whatever you want. So go ahead and dig straight down, but don’t blame me if you fall into lava.