Nope, you don't need to have an account to place an order with UncommonGoods. You can simply shop, add items to the cart, and check out. When you place your order, you can provide an email address for order confirmations so you can track your order. Doing so won't create an account and it won't result in a subscription to e-mail newsletters unless you choose to subscribe.
Having an account is pretty handy. You'll be able to see your order history, any wish lists you've created, and the shipping addresses you've used will be saved for future purchases. An account can be created before you start shopping by clicking the "create my account" button (image below), before you check out and place your order, and immediately after your order is placed.